n 160 jl05 

kOTBOB 
f ITLB 

IMSTMOTIOM 

PDB DU?E 
VOTE 



IDFS FRZCl 
BISCRIPTOBS 



IDlHTIFIlRS 



OQCOIIR^ IBSIIE 



ia» * 

Intastlgations in Saiesca E 
Eipanded ib straps an^ Crlt 
H^MareS* , * 

Ohio Stat€ Onl7,, CQluibua. 
Ha thmm aMcs . Edu cat Ion . 
77 

65p, ^ ^ ^ 

inforBation SefereEca Ceote 
State UnlT, , 1200 / Chai ters 
.OH a3212 (aubacriptleE 16.0 

MF-$0*83 HC*i3*50 tlum Pest 
♦ Abstract si *lkutoinatriictic 
Scl€iicQ| CarricQlai Devilop 
Besearchi •Evaluation ; ^nt 
*Sci©nea Educatloii; Sci€ntl 
Schccl Science; *studeBt Ch 
Reafawh Bepotte 



^ SB 02« 973 

dQcatice , .Vol.v J , uom 3« 
ical Anaif^ei c£ B€c€nt 

Center for Science and 



r (IBIC/IRC) , The Ohio 
Bd. , 3rd FlcQr, ColUBbua, 
0, t1«7S single copy) . 

age* 

cal Jethcdai College 
■entf ' EdncaMcnal 
trttcticnal teaign; 
fic Conceptsi Sicondacy 
aract€rli|ici ; Tests 



IBSTBICT 



jr ; ■ ^~^Thia Issne contains expan 

reports qroupsid into tuc clusters and a s 
stddi^s« The first cluster contains atstr 
dealing with trait- treatment interact icn 
deals vith ejcaBinatldn items categorized 
Li%xOMmf, The final ^ction> individual s 
/stud^s on the following topics: (1) stud 
instructors; (2) science aisconceptlcnsii 
and (U) self *direc tedness. Each abstract 
rationale^ research design ^nd pTOcedrre^ 
and an* abstractor* s analysis of the resea 



ded abstracts^ cf research 
ection of individual 
acts of tvo refetrch report^ 
studies* The aecond dusker 
according to BlooB*a ^ 
tudleaf p^npludes f^ur 
est evaluation of 
(3) instructional design ; 
Inelttdea purpose, sesearch 

findings^ interpretationspv^. 
rch* CGI) 



♦ Reproductions supplied by EDBS are the beft that can be made ^ 

* ! from the original dccument* ^ * 



EKLC 



IiTVES,Ti)^TIONS IN SC-IENCE EDUCATION j * 



0 



Editor ^ ^ 

Stanley L, Helg^spn 
The Ohio State 'Univars:yty 

' Agjocjata Editor 

Patricia E, Blossei 
The Ohio State Unlveritty 



U i OePASTMlNTdF^NEALTH, 
EQUC&TION ft WELFARl 
NATiON&b iNlTITUTf OF 
iDUCATIdN 

THI| DOCUMtNT HAS BEEN HiPRO- 
OUCED EXACTLY AS RECilViD F^OM 
THE PERSON OR ORGANIZATION ORIGIN^ 
ATING IT POINTS OF ViEW OR OPINi^ONS 
STATED OQ NOT NiCiSSAgiLY REFRE- 
5ENT OFFICIAL NATlONALMNSTlTUTi OF 
EDUCATION POSITION OR FQLlCY 



David P. Butts (1978) 
University of Gaorgla 

Kennetl G. Jacknlcke (1978)' 
University of Alberta 

Donald E. Reichard (1979) 
toory Univer-slty 



Advigory Boarc 



Ronald D. Anderaon^ (1981) 
U^verslty. of Colorado" 

Franees Lawrenz (4980) 
Minneapolis, Minnesota 

V 

Joe C, Long (1981) 
University of Georgi^ 



Kational Association for Research In Science Teaching 



ERI5^^1earlnghouse for Science, Mathematics , 
^ and Envlronmant/al Educatlori 



Published Quarterly by 



The Center. for Science apd Machematlcs Education 
College ofi Education 
The Ohio Scace University 
19^fr5 NorthlHigh Scraec / 
^ Colginbus, Ohio'43210 ^ 



1 ^ 
Subscription Price: '$6.00 perj year. Single Copy Price: $1.75, 

Add 25c for Canadian mailingsj and SOc for foreig^^ mailings , 



from tha Editor 

f 

. ^ ^ ^. ^ ^ ^ — ^ ^ ^_ 

This Volume contains research^reports grouped into two cluiters, 

and a section of individual studies* The first cluster, TMIT-TREATOENTs 

contains analyses of two research reports dealing with trait- treatmant 

Interaction ptudles. The second cluMerj BLDOH'S TAXONOMY, Includes 

analysei of^ studies ^dealing with examtnation Items c&cegorlged accord- 

Ing to Bloom's Taxonomy . The final sactlon, INDIVIDUAL STUDIES^ tnaludes 

four studies of such diverse topics as st^ 4gng evaluation of Instructors^ 

science misconceptions , instructional design, and self-dlrectadness , 

Publlshabie responses to these analyses or coiMnents and suggestions 

for improvement of the Journal are invited. 



/ Stanley L. Helge$'i^n 

\ Editor ^ ^ 



Patricia E, Blosser 
Associate Editor 
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Purpose 

Bie general purpose of this investigation was twofold. It was designed to 
evaluate two methods of Instruction in a physics course at Cornell Univer- 
sity, The study also examined two procedures for asslgnteg students to 
these methods. ^ 

The two n^thods of instruction co^ared were audio-tutorial (AT) and 
leetu^=recitation-laboratory (standard). The two procedures for place- 
ment of students were random assignment and assl^ment according to 
student preference. 

^e investigators also sought to compare ImEoeftate and longer-term effects 
related to both achievement and attitude among the four treatment groups. 
Data from the 1974 study were compared to data from a similar investiga- 
tion conducted in 1973. , 



Rationale 

Physics 112, Mechanics and Heat^ Is a one-semester introductory course 
offered to approximately 550 engirl^ering and physics majors eacl\ spring 
at Cornell University. Most of the students are freshmen level imles , 
Faculty working with this course ware interested in comparing achievement 
and attitudes betwean two methods on instruction: AT and standard. Their \ 
Interest j however, ^pnt beyond merely seeking to detarmine which mathod J 
was superior. They were also interested in learning which mcathod was 
more suitable for certain students. They were interested in comparing 
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achievement and attitudes between students randomly assigned to the two 
methods Bud allowed to Enroll acGOrdiiig to preference. ^ Other 

variables such as Scholastic Aptitude Test (SAT) scores, mathematics 
achievement pre-test scores ^ and attendance records were compared across 
groups In what the investigators termed a "trait-treatment-interaction" 
approach. 

In shorty the rationale for this study was to learn W different types 
of students at Cornell "^University performed in a differential mannfir 
when itudying introductory physics via two contrasting methods. Like- 
wise ^ there was interest in learning if an interaction existed between 
student. attitudes and the uistructlonal methods us%d. Furthermore, there 
was interest in eKploring other student traits in light of their*cogni= 
tive and affective behavior. 



Research Deslgnand Procedure 

At Cornell University approximately 270 students faking Physics 112 ^ 
Mechanics and Heat, were assigned to^our tteatMnt groups during the 
spring semester of 1974, The treatment groups related to the methods 
of Instruction and to the procedures of assigning students to_ the methods. 
Hence J the four groups were^ audiQ--tutorial-random (ATR) , ^dio--tutorial- 
preference (ATP), standard-random (STR) , ^nd standard-preference (STP) . 
Students in all groups had the same textual materials , rtie same homework 
and laboratory assignments ^ identical quizzes and examinations, and 
roughly the same content. The standard method included^ two hours ofti 
lefiture and two hours of ^recitation every week and a two-hour laboratory 
period every Other week. The /AT method included one hour of recitation 
per week and was designed primarily to allow group interaction as well 
as s^jadent contact with one particular instructor. All other instruction 
with this method took place at the studenG^s convenience in a learning 
center staffed by tutors 4 7 hbura per week, f^terials in the learning 
center included apparatus for ' self-depionstrations, the same selection 
of laboraC-ory equipment available in tha standard laborp^cories , audio- 
tape commGntaries and slides. These mnterials emphasized concept develop- 
ment as well as problem solving and were used in addition to the course 
textbook and supplemc-nCary notes, \^ 
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Studenti ik tKls stud 3% were assi^ed to 1 of 15 sections scheduled during 
four differant class hours. (Students were enrolled in other sections 
but were not used in th'is study*) Eight of these sections becaDte the 
standard group and seven becane AT recitation sections. At two of the 
four class hours, students were allowed to choose the treatment based op? 
thed-r preference of teaching methods, ^ a result of further random 
assignment J there resulted four ATR sections, three ATP sections and iow^ 
sections each o£*STR and STP, The 15 recitation sections In this study 
were taught by 8 teachers. Including 2 faculty meinbers and 6 graduate 
students, Severf^ of the teachers taueht one AT and ona standard section 
each. One "gTaduate student taught only one standard section Included In 
this study* " ' ^ 



To Investigate ^possible differences between* the effects of the methods on 
different student groups, the Johnson-Neyman technique oi regression anal-- 
ysis was used. The author stated that regression analysis was used 
Instead of analysis of variance becau^ the latter rlduces continuous 
scores to a sBmll number of levels, makljig this procedure inefficient,. 

The basic measure of student achievement in this study was final grades. 
This variable was based on recitation performance ^ laboratory reports, 
two preliminary examinations and a final examination. The "traits" ^ 
analyzed ^in this study were achievement in mathematics as measured by 
,a course-specific pre-test and mathematics apt^^tude as measi^red by 
students' scores on the mathematics portion of the SAT, ^ 



Student attitudes toward the AT and standard; methods were measured by ^ 
responses to a questionnaire completed by students at tfteir last reclta- 
tlon Meting of the semester. Two "fairly gl^obal statements" were 
included in each questionnaire: "In general,^ I have been satisfied "7 
with the AT (standard) method of instruction used in P112" and "I am 
glad I took the AT (standard) version of P112 rather than the standard 
(AT) version." \ 



Longer range effects of the differences in methods of instruction were 
,=considered in this study* This was accomplished by analyzing the pro- 
^^ress of students in a similar 197 3 study, , The investigators sought to . \ 



determilie (1) whether studeats from the four treatment groups enrolletf 
In fch^ same tjrjes ofl^n^lheering courses, (2) if there were ^ny nega= 
tlve effects of thm AT mathod wjien students returned to a lecture- 
recitation-lab fiormt in/their science courses,* and (3) whetHftr AT and 
stMdard studat^s had different attrition rate% in later semesters, ^ 



Find ings - ^ ^ 

— , J 

The design of this study (1974) was based, at least in part, on results 
by the same investigators from a similar study conducted during 1973i 
The results of the earlier research indicated that an interaction existed 
between two student "traits'' (mathematiM .achievement as measured by a 
eo^rf e-specif ic pre-test and mathemtlcs aptitude ^s measured by^ the SAT) 
and the two methods of instruction, Students with very high mathematics 
aptitude (SAT math scores of 725 or high€r)^^nd high mathematics achieve- 
ment on~ EH e pretest had higher predicted' grades in the standard method 
than did comparable students in the AT method * Students with relatively , 
low mathematics aptitude (SAT math scores of 625 or lower) and^low mathe- 
matics achievement had higher predicted grades in the AT method than did 
their counterparts in the standard method » Using the same linear regres- 
sion tecl^ifiques, predicted grades of students ranking intermediate in 
mathematiW^_a£titude and achievement did not differ significantly in the 
two methods. Additionally*, in tHe 1973 study there were differences in 
course grades within the AT method between the randomly assigned students 
and the stiidents assigned by preference. ^'Jhen matheMtics aptitude and 
achievement ware controlled, AT students who hadTbeen Assigned randomly 
had significantly higher achievement than did those assigned by prefer- 
eftce , The 1974 investigation was, therefore, designed in part to deter- 
mine whether these 19 73 findings would be reproduced. To this end, 
previous results were not revealed to course instructors of the 1974 
study and attempts were m^do, Co maintain similar content coverage across 
years of treatments. 

Results from tha 1974 investigation domonst racod' no significant difference 
between the regression lines of the AT and standard groups when predicted 
grades were relatod to mathematics pretest and aptitude scores in an over- 
all comparison bctwoon mcj^ods, 
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>fajor f Indingy f rom the 1974 Investigat^n revealed that student achieve- 
ment difference existed* between the VBn^gm and preference assignmtn^t 
prDcedures, Amo^ studei^ts who were randomly assigned the methods, 
mathematics achievement was significantly related to grades in each 
lEiBthp^ of* instruction. The slopes' of the regression lines relating 
pretest mathematics achievement to' predicted physics grades did not dif fe 
significantly in the two methods. Mathematics aptitude as measured by 
the SAT, conversely, was not highly related to grade in either method 
for randomly assigned students. Investigators ^ therefore, concluded 
that there was no evidence of a trait -treatment --inter act ion among the 
randomly assigned istucjents, ^4mong students assigned by preferenc^ y how- 
ever^ dif ferences ^d exist. Those who selected the standard method 
achieved higher flnal'grades than did those who selected AT, \^mn pre- 
.dieted grades J^ere related to both mathemtics achievement and aptitude^ 
the predicted grade was higher in every case for tKe ^.^P group (over the 
ATP groHp) when the Jjiiinson-NeyiQan technique was lised. In particular, 
for arrange of SAT mathematics values of about 630=670, the Johnson- 
Neyman analy^s predicted signlficmtly higher (at 0,05) grades in the 
STP group, ^ y ^ ^ 

Student attendance patterns were different between the ATP and STP groups 
Seven of 45 ATP students responding to a final questionnaire indicated 
they stopped afitending the learning center on a regular basis by the 
fourth week of the course. None of tlc^ 74 STP students indicated on 
this questionnB-ire that t\\Bf had stopped attending lecture that early 
in the semester. In addition, 24 of the ATP students said they did not 
regularly attend the. learning center. STP students repbrted spending 8.6 
hourl per week outside of class while AifP students reported 5, STP stu- 
dents exhibited a considerably higher average grade on the first exam 
than ATP students while grades on other tests did not differ signifi- 
cantly, ATP students also possessed slightly lower grade point averages 
in their other courses when compared to STP students. 

When students in the 1973 and 1974, studies responded to two Likert-type 
attitude ataternGnts ("In general, I have been satisfied with the AT/ 
standard method of instruction used in P112" and am glad I- took the 
AT/standard version of P112 rather than the standard/AT version,'') ^ the 
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following results wer| obtained" (1) students assigned by preference 
,expresaed more positive attitudes tcward the method they re'ceived than ^ 
did students assigned randomly; (2) AT and standard students tended to 
be equally satisfied with the method they received; and (3) standard 
students, in/1974» but not , In 1973^ were more likely than A*/ students 
to say thA they wer^ glad to have take^ thalr method of instruct iohs.^^ 
rather than the other method^ The author has been quick to point out, 
however p that Increased disaffection with AT ^ould well have be6n due / 
to^ problems associated with "coordinattog the two metho,ds of instruc- « 
tion," ' - ^ 



Fq11ow--up behavior of approKimately 90 percent of the students In- the 
1973 study was analyEed, AT , and standard students. enrolled In roughly 
the same selection of engineering courses subsequent to the 1973 investi- 
gation. Consldetin^ students- grad|fs in sophomore engineering and physics 
courses J ATR performed as well as, and in some cases better than, STR 
students, STP students , on the other hand, performed somewhat better In 

some courses ' than their ATR counterparts* While students in ATR outper= 

_ ^ - - -1 

formed those In STR ar\fl STP outperformed ATP students in other courses ^ 

when taken together tthere were no apparent overall differences in achieve^ 

ment between the two^teaching methods. Comparing attrition rates *among 

thfr fou^r groups produced a similar pattern^that of attrition being 

inversely related to achievement. Again ^ howeve^ when the two methods 

were compared^overall therre appeared to be no sl^if leant difference 

between standard and AT instructional groups. 



Int ei^pret^tlons \ 



One of the purposes of this study was to compare student achieivgjnent of 
st^udents at Cornell University enrolled In a physics course taught by 
two contrasting meth&d^- of instruction: audio-tutorial and standard 
lecture-recitation-laboratory. The most significant finding of this 
investigation was the fact tla^at wlpn students were assigned ^o the two 
teaching methods by random ^rocedul^Sj there was nb difference in achieve- 
ment between treatment groups, \Ti\^n students J^ere allowed to select the 
instruct ional method based on preferen^^ however, the group taught by 
th^ standard method e.^liib^ted supe rio-r eve ment in physics** 

y 



Con|rderlnfe/aitltud^s alongside achievemerit, results from this study 

Indicated znat students assigned by preference were more ratisfied with 

the method they i^epeivfed than were the randomly assigned students 

(althbugh, AT students were more* likely than standard students to say 

they would have preferred the alternative method) , Fol|.ow-up studies 

did not indicate any appreciable difference.^ in enrollment pa^arnss 

* / - . " 

course grades, or attrition rates .between students taught by the two 

methods, ^ * 

The aforementioned sumnary suggested "to the inveitigators of the report 
a dilemma. If students are allowed to choose between two methods of 
instruction such as lecture-recitationwlaboratory and audio-tutorial, 
they are allowed a greater degree of flexibility and self-determination. 
Results, from this study suggest that-fl-this leads to greater student aatis= 
factionj to more positive, attitudes toward the method of instruction in 
which students are engaged^ In the case o^ the students in this study' 
who were allowed to take AT physics because they preferred it, however^ 
a lesser degree of achievement ensued. One reason offered for this was 
/that the students were apparently overconfident . These students as a 
group had tak^ more^hysics in high school and exhibited somewhat higher 
mathematics aptitude as measured by ,the SAT. Additional data showed that 
they spent less time^ studying out of claifs and that they were more likely 
ta stop attending the learning center regularly. The investieators 
offered no solution to the dilemina. They did suggest, though, that 
additional^ indicators of student achievement and attitude need to be 
explored in relation to teaching effectiveness. Gains in such consider^ 
atiiDns as studeni^ indepandence, self-conf idance, interest in the subject 
matter and desire £o take additional physics courses would certainly 
represent fac'tors that might tend to mediate the results produced in 
this study. ^ ^ V 

. ( • , , ■ * 

ABSTRACTOR '"s ANALYSIS 



One of" the' most frequently asked questions of college l^el instructors 
is "Wlilch Eeac^iing method, or methods ^ should I use in my classroom?'' 
Among the mist frequently discussed altdrnative teaching strategies in 



ighcr educatiion is the auto= or audio=t: utor ial ('AT) ^mauliod* SomG dis^= 



pr^y of this approach tfrl modif icat ion thereof can-/De found somewhe.re^ 
within most colleges and yniversities , One of the* most ^tempti^g areas 
of edticational rpsearch is that of comparing s^tudent parformance among 
a variety of teaching approaches,'- Since, represents a rather dramatic 
mii^ visible shift in American pedagag^, it is only natural th^ college 
'level educators Xy/ould ba. interested in coinparing it with tyadit.ional 
me f hods , The li td'rat ure has indeed containad many such st udias ovfir 
the last decade* Yesfj in most cases results have been cloudy and incon- ■ 
elusive* And, when r'asults from one s'tudy sre compared with those of ^ 
another, it is usually dif f^-qlt to formulate substantial |eneralizations* 

One 'of the reasons it is difficult to draw conclusions from methodologi- 
cal studies of -the type, just described is that within any Investigative 
setting exist variables which are next to impossible to duplicate. This * 
will likely "Remain a drawback to educational research of this type. But, 
there is another traditional weakness of studies of this type and it is 
that the dependent ' variable generally consists of a single measure— that^ 
of factual recall of material on standardized or teacher--made tests* 
The potentially powerful effects of an alternative teaching method is 
--pften never measured when a single measurement ' of achievement is used, 
Ther^ is at least a third reason wiiy educational studies of this type 
are often inconclusive or meaningless* They do not 'consider the nature 
.of the students They frequently, f^il to oonsida^r the fact that indi-- 
vid^al students react very differently to the same stimuli, simply because 
each student has different neads^ different attitudes, different cQgnitive 
styles and different levels of maturity. 

When considered within the matrix of other investigations of this type, 
this study rap resents an advancement in^^oth knowledge and research 
methbdology. New ralationships have been for^^arded here with respect 
to Kpw additional studant variables or^ '^traits" rnay meditite aekievement 
and attitude* Furtherfaore , the use of such techniques as preliminary, 
follow-up, short ^ranRG and longer-grange measures, in both cognitive and 
affective dimenhi.LonH / represent potential paronv:^tnrs in which new rcJ.a- 
J^on s i 1 i p s c an b e d i s c o ve r a d , Had 1 1 1 is s t ud v b a e n in vo 1 d wit li me re 1 y 
i gn in g s t ud o n t h to t w o t r a a t n i eii t g r o u p s and comparing c ou r s e 
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» achicvamcnt ^ no signiricaat: ciiCforcnced would likely liava cxiHtccu But 
the research dakif^n in this t^tudy aliowud for the construct '* preference" 
to be considerGd; and, as a result we loarnod that there is an apparent 
difference in the way engincGring students at Cornell . University perform 
in Mechanics and Heat wlien they are allowed to choose between two teaching 
methods. Likewise, in^ measuring such variables as attitude toward the AT 
Btethod of .in^.tructiony attitude toward the lecture method^ attendance, 
attrition, hours of study per week, subsequent selection of courses, 

^^umber of physic's courses in high school, mathematics aptitude, and other 
pertinent variables, it is possible to compare the two teaching methods 
within more than one context. In essence, this study avoids a trap o f ^ 
oversimplification. Ins ^ ad' of being forced to'^hoose between two teach = 
ing methods— to' declare one the "bea.£^-— educators and researchers .are 
free to examine several relationsj^iips that emerge from this investigation. 
Further^, there is something concrete on which to build other istudies . - 

J' 

Would an investigation of this design produce similar^' results at other 
institutions? Since engineering and physics students, Mechanics and 
Heatj and faculty personnel at Cornell University are no doubt unique to 
other settings, the external validity of this study has not been estab = 
lished. Reasonable safeguards ware taken, however, to build a case for 
internal validity. The regression analysis procedure used in this study 
appears to be an appropriate statistic. The author of this paper has 
been quick to point out potential weaknesses and concerns. One such 
factor related to the heterogeneity of the student population in the 
1974 studVp. Apparently^ many higher ability students took Physics 112 
<iuring the fall of 1973, leaving perhaps a skewed population during the 
spring s e mo s to r o f 19 7 A , In o r de r t o e s t ab 1 i_ s h e x t e r n a 1 validity th is 
study should be repeated at other institutions with other student popu= 
la t ions. ' . 

I found this report well-written and easy to follow. It presented a 
rationale that was defined clearly and logically. The sample and pro= 
c e d ur e s we re d o c u me n t e cl an d p e r t L n e n t da t a w e re displayed. In practically 
all casoSj in formnL Inn in tlio tables and f|i.;^ur'es dt^^^eloped adenuatoly 

in the body of the paper* The results and concluHions were stated 
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carefully and congrucntly . The reasoning used to develop relatlonshiplj 
among the variables was easy to follow an.d underscand, ' ^ 

Research of this^ type is always difficulti and it is not ^always possible 
to coAtjrol all variables to the extent one would preier. As stated 

, earliey^ the author cited several things beyond her control in this, 
ras^arch design. , Perhaps the most noticeable ladk of control came when 
a ^change in professor decreased the degree of coordination dn the content 
of th^' two methods and" between the 1973 and 1974 studies. In fact^ the 
variable **professor" or "prOj^essors" ' Constitutes a powerful treatment 
effect in studies of this type and, when not controlled, can ^ead to 
some hard-to-answer questions. Tlie '^like" or ^'dislike" by students of 
a" key faculty member or two in a study like this (especially one with a 
"small ^) could mask attitudes . toward "teaching method," In a couple 
other ' ins tances 5 it was difficult to see whether data from, the 1973 study 
could be rightfully compared with data from the 19 74 study. 

My biggest criticism of the study is that so few affective measures were 
tatesa*^--^ It is quite possible that students were focusing their :fHe lings 
on objects of affect other than "teaching method," Attitudes toward 
^*pUa(aicsff?L.*' teacher^" "college," "engineering," "studying," ;*grades," 
, and "academic self" represent areas that could be more important to frash-*^ 
man level students than "Aff" ^^r "lecture," Of course, the author ailuded 
to this at the end of the report. She also mentioned variables such as 
student independence and self-confidence, which Indeed could be signifi- 
cantly Influenced by a teaching method s'uch as AT which places more of 

p. . . . 

the responsibility for learning on t^e^ studen^t . ' I^lle . students often 

insist that they prefer "flexibility" and "freedom," it is not surprising, 

that they might not only perform better but actually feel more secure 

with a ■ traditional teaching method^ one under which they had been pre= 

vioualy nurtured, ' ^-^5 

\^ I think this Investigation can serve as a model on which to build further 
studies of this type. By introducing and controlling for additional cog- 
nitive and affec^tive vari/ihle55, additional rclationsli ips can be soughs. 
By us in g re g r e s s i on ana 1 y s L s 1 1 i s p o s s i b 1 o t o dc ve 1 o pi r e s e a r c h de s i gn s 
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that allow rayoarchtitrH to cxamino crucial fact'Srs as both independuiit 
and dGpcndGnt variables.' For /axamplG , variables sucii as "matlienfatics 
achievement** and '*acatlemlc saa f -concept-' ai^e botli entry traits as well 
as exiting traits, Tliese and other variab^s not only influogce achieve 



ment but are influenced by achievement. Students possess these traits 
when they come to us and they pos^ss them whenr they leave us* Often 
the effect of one course is not power fcul enough to induce significant 
change g Studies of this type should include long-range components. 
For instance J the effects of one AT course may not be signif leant ^ but 
several courses where students ^must leaim to be more independent may 
cause a^thange that could be measured later. Iti is impdrtant to remembe 
that one course or one professor or one teaching method i^s but a small 
portion of the total educational experience of a student. 



As more studies like this phe are conducted^ educators will slowly un- 
_'cover additional rel-^tionships central to the teaching-learning process, 
this happens we will surely be able to improve the instructional 
setating and acadeniic performance of our students^, rhis study is an - 
example, of how a carefully planned ^ carefully conducted, and well-- 
reported investigation can help expand the body of knowledge in college 
teaching. 



/ 
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Ott, >tary Diedcrich nnd navid B, Mnckl in, "A Trait-Treatment Interaction 
in a Collcj^e Phvsics nourse,'' of_Rt:-searc h in Scioncti 

Teachtnp, 12(2)* ^ Ul-119, 1 975. ""^ ~~ 

Descr Iptors-^-'-AuLolnBfruc t lonal Methndst. ^'^CallGge Science; 
Educational ResGnrch; Hie^hcr Education; Instruction* -Physics; 
*S Science Education' -Student Charac ter is tits * -Teaching Methods. 

i? ^ - ' 

Expanded Abstract and Analysis PrGpared^^ Especial ly for I,S,E, by Dorbthy 
Gabel, University of Indiana. , "'"^ , ' 

\ 

* J - 

Purpos_e ■ • 

e 

The purp^ose of this study was not to determine if aud-io-tator ial V._J^ 
instruction was superior to conventional iristruction but to deterTninG 
if an interaction. between the type of instruction (audio-tutorial or 
conventional) and studenns' traltr^ (Tnathematical ability) had an effect 
on achieveTnent . 



/ ' ■ 

Rationale 

Frequently no significant ^ dif forences a^e reported in student achieve- 
merit due*' CO differences in instructional strategies* In this study the 
authors follow the suggestlcSns of Berlinger and Cahen (1973) and Cronbach 
and Snow d969) to examine Interaction between student traits and 
. instruct lonal merh^oEs r S^^ch ^ findings should enable educators and students 
to select instructional strategies that enhance particular students- , 
achievenient , 



Research Design and Procedu r e 

,The sampie consisted of 303 college engineer ing .and physics students 
(98 percent freshmen) enrolled in a one semester introductory level 
pffy s i c s c o u r s e . Stud e n c s wh o ha d a s c he d u led recitation per io d at a 
given hour were randomly assigned to a class section Vhich had either 
audio-tutorial or traditional instruction. In about half the cases, 
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however, students wtrp^ allowed to tndicntio n prcfercMHic fur a parttcula,r 
type of instruction. .luT ihe&,e caseS the ^^^ttidonts weru rnndomly rfsslpncd 
J=o brre of the sections with their preferred mndcal of inBCructiun. This 
resulted in 115 in the audio-tutor inl treatment (5 7 random Iv ^as signed 
and 58 selecting it) and IBB in ^e standard treatment (101 random, 
87 p^rferred) in each. These students were distributed in 15 recitation 
sect ions . 



^^he treatment consisted of instt^nction by^- two dif f erent niodeSj audio- 
^Jutorial and conventional. The audio-tutorial method included one hdurl 
of recitat ion per week, All other instruction took place at the. s tudonft * s 
convenience' in a learning center staffed by tutors 52 hours per week, ' 
The^ standard or convent ionnl method included two hours of lecture and two 
hours of recitation per week and a two hour lab every other, week. 

Course content in both treatments was comparable. Both groups had the 
same homework assignments, sim.ilar lab experiemtns, and the same * 
examinations* The lecturer of the standard course was the co--author of 
the^^ater ials used in the audio-tutorial course. 

In order to determine students' original traits, students were given a 
questionnaire and a math pretest at the first recitation period. The 
authors then selected traits on Che basis of (1) the im.portance of the 
tratt to the course, (2) the reliability and validity of the measurement 
-"'of the trait, and (3) range of^responses to the trait measure. Three 

traits met the criteria: the College Kntrance EKamination Board Scholastic 
Aptitude Test in Mnthcmatic^^ (SAT'Tt) , verbal aptitude of the same test 
(SATV)^ and a mathematics 'pretest com.posod by the authors, SATV was 
ellmihated because it was foi_/nd not significantly related to the final 
grade in either treatment . 

The d ep e n d en t v.^iab 1 e , student* s a c \\ i e vemen t i^ t he c ou r se ^ wa s t hen 
me a s u r e d b y the f i na 1 . c mi r s e r a d e , This wa s based o n t h e s t u d en t * s 
lab work, quf:^zes, interim examination and final examination, 
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data were analyaed in Rcvernl ways, Mt^ans and suand/ird deviations 

of the SAT>tj Math Pretest and Final Grade ware compared for the two method 

of Instruction u^in^ a t--test (not eKpliritlv stated). Regression 

equations were formulated to predict estimated grades resulting from the 

two instructional modes. The Johnson and Ne>nTian technique (1936) was used 

to analyze the interaction between the tEait (mathematical ability) 

and treatment (method of instruction), 

. ■ ■ ■ ( 

Find in \ 

Although there, were some significant differences in pretest math scored " 
and SATM variances at the 0,05 level, there were no significanrt differ- 
ences ,in the means or variances of final grades in the course. 

The following regression equations v;cre formulated! 
Audio-Tutorial Z ^ 0,5175X^ + 0,2633Yi + 9,1667 
Standard A - 0,8495X2 + 0.4081Y2 +10.6352 

where Z and A are course grades, X is the score on 
the math pretest and Y the level on SATM, 

The Johnson and 'Icym.an technique Indicated that at the 0.10 level of 

significance the standard treatment was preferable to the audiO" tutorial 

in terms of expected achiev^ftient for students with math pretest scores 

equal to 9 and SATT! greater than or equal to 725^1^^ The audio-tutorial 

trcatmen'^ was preferable for students with math pretest scores less than 

or equal to 4 and SATil less than 625, Both methocjs of » instruction were 

\ 

equival>ent for all other studentf, 



^n_ te r p r e t a t i oji s 

Results of. this experiment indicated that although there was no signtficanc 
difference? i n^ph y s Lc s a c h 1 e v em en t for student s who studied using 1 1 1 e a u d io 
.tutorial method of standard methods, there was a sipnific/int interaction 
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between physics achtoveTnent and math skills as mcasurod by a pretest and 
the SAT>t, Studcrits with high ability profit more bv the conventional 
treatment whereas students of less ability profit from the aud io-tutoriai 
method. Because this study is liTnited to a specific physj.cs course, 

results cannot be genei^al ized . SiTnilar studies should be 5qnducted to 

~~~ 

determine if this trait-treatment interaction is applicable in other 
courses, i 

ABSTRACTOR'S ANALYSIS X 

Although this study has limited general izability, great care was taken by 
the authors to randomly assign students to treatments and to control medlat-^ 
ing and extraneous variables. In addition, the s^tudy^ is concerned with an 
area of research^ trait-treatment interaction, that is of growing Interest 
and concern t^science educationp 

There are several areas in which this research and the report could be' 
strengthened. First; the purpose of dividing students into two groups^ 
those that preferred a particular mode of instruction and a randomly 
assigned group, is not clear. The authors do not justify this classifica- 
tion In their rationale, do not state hypotheses concerning itf and do not 
make sufficient use in their analysis. These data probably could be further 
analyzed to compare differences in achievement between students who selected 
audio--tutor ial instruction and those who were assigned to it, 

A second concern is with the validity and teliability of the instruments 
used to measure the students' Mathematical skills. One of the two instru- 
ments used is a nine-item muTtiple choice math pretest. The authors state 
that the correlation between this test and the SATrf test, they administered 
was 0,33 and thprofore appeared to* measure different traits. Although 
this may be true, no menti'on is made about the reliability ot this test, 
A nine-item test may have ''low reliability and^^^theref ore invalidate\^the 
study, . i 
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Another area that noods clarification in the article in lu the intcrpreta=- 
tion of the tables* Althcrmgh" number of subjects in Table III is listed 
as 3P3 students, the number In Table. II lists 282 students. It is also 
difficult to Interpret the means in Table III, The dependent variable is 
ed as the coi(rse final grade based on scores ^hat range from 0 to 400, 
The reader really needs to know how these scores are, translated into grades, 
A final grade of 6,44 has little meaning as one does not know to what letter 
grade it is equivalent 'or how it was^derived. Because of this, it is also 
difficult to interpret the regression equations that ara given for both 
treatments* ^ 



In addition, specific mention should be ^de of which statistical tests are 

-- - 

being used. One could probably as^^ume that a t--test is uied for the analy- 
sis of the means in Table III but ^no mention of which correlation coefficient 
\^aB used is given li^ the article. / 

^^cont^bution that this study makes to this area of research is in the area 
.of m^thdyology , The methodology for determlnlng^the trai^-treatment inter- 
action appears sound;t^^ The method used was thei Johnson^NeyTnan technique, 
By^Ftislng this technique one can^t o»ly deteriiyine whather there is a slgni- - 
f leant interaction but also the level of the trait/ that will yield^ a " 
>l-talf leant Interaction at the level of one's ch/ifce. Other resea^cheM^' who 
investigate.^ tralt-^treatment interaction may wish to examineithis methodology 
to- detertTLlne its applicability to their own studies, 

' J ' ' ' ^ 

Studies of this nature make a significant cont^ibut ion ' to science education 

even thougl^^ the results may not be generalized %o other courses. This study 

acts as a^^model on which replication studies in^ther educational Settings ^ 

caf/ be carefully conducted. By combining result^. 5rom. a series of the^&e~^~" 

replications, generalizations can be made on successful instructional 

strategies for varidu^^ types of students in diverse settings* 
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Bill-ch, Victor Y, ='"An AnnlvHls of Teacher-Made SciGnce Test Items in 

Light of th.e Taxonomic Objectives of Education.^'- S cience E ducnt ion , 
58(3)1313-319, 1974t __ 

Descr i.pCnrs-^Achicvement Tests* Educational Research; ^'^Evalua t ion| 
Instruction; '^Secondnry . Schno I Science* feacher Characteristics; 
Test Construction; -Tests ^ 

Expanded Abstract and Analysis Prepared Especially for I,S,E. by David H, 
Oat, California State College, Bakersfield. 




The /study is described as having two major purposes I (1) to identify the 
patternfj.of the cognitive processes' Implied in teacher-mada examinations 
in secondary school science; and, /p)/' to relate the pattern to selected 

variables (i.e., grade leveli subject ma t ter = taught , and some teacher 

' " 

characteristics). From these two m^jor purposes, four questions were 
asked i 

1, What relationship, if any, exists between the level of the test 
^Item as cJ^tfssified by Bloom ^s Taxonomy and each of the following 
teacher characteristics i years of sciencel teaching experience, 
professional in--service training, academic specialization, and 
status as ipart-time versus full-^time? 

2; Is there a relationship between the science subject taught and 

the level of the test item? 
/ - • 

3, Do teachers of di|ferent grade levels emphasize different cogni-- 
tive levels of test items? 

4, Wtiat percent (proportion) of the test, items asked by secondary 
school science teachers falls into each of the categories of 

4 Bloom's cl Asif icatipn system,? 

\ ' ' 

Rationale 

The relatively recent attention piven to the types and strategies of Hsk-= 
ing questions has yielded cons i flernble insiplit: into the comploxitias 



^Bllleh does not e:<plnln whan he means by a pattern. He docs not 
appear to discuss the results of his research in Cf3rms of patterns. 



^of the proctiss. For example, it has hei2n shown that through proper 
questions,"' the .teachur can havo a iagniflcnnt [influcnco on diroctlng' 
and developing the cogniclvi:' processes of Btuddrits, Test items are qu^- 
t^ciTs and must be included in these types of studies. Questioning, in s^i^- 
■ the form of examinations/ is subject to the same forTns of anMysis. 

It is therefore necessary that questions be classified in a manner which 
identifies the level of cognitive process ne^cessary to develop an appro- 
priata answer to the question. Bll|^h sug^sts that in determining 
xthe c^nitive level of a particular test item, 'the judgment must be/€ade^\ 
In the context of the learning exper iencfe-S , instructional mater ia!^ 
4nd othfein factors to .which the students were exposed or which were usld 
to stimulate learning of the material represented by^ thk test item'. 



iea rch Design and Procedure 

Twentyrtlve randomly selected secondary schools in Beirut, Lebanon^ 
which^^^^ at least grades seven to ten were identified for the study* 
One f(fls3B each of grades seven and ten was selected in each school, 

Tapej recorders were use4.^^o obtain a verbatim record of three-to--f ive 
hour s&eslons covering one science unit in each 'class* Each teacher 
participating in the study was asked to develop an hour examination of 
the unit which was recorded. Thirty- three examinations ware submitted. 
In addition. Information about the teachers' academic specialization, 
teaching expor ience , ^t^ining ^ and status was obtained, ' 

w 

Three j u^«"^xper ienced in Bloom's taxonomy worked independently on 
classifying the test items according to^ Bloom-s classification system, 
Consensus was later reached as to the cognitive level of each tes^ 
item. The percent of test items falling into each level was calculated 
on using the weight .^ssigned to each item by the teachor. The m.ean 
proportion of test items representing each cognitive level was 
calculated. 




Use of one-way analysis of variancea was made to analvze tho rclacion-- 

ship between the scioncu .^ubjijct taught and the level of the test item* 

f 1 
The t-test was ust^d to determine whechp-r cognitive levels differed 

-^^lEwe^n grades seven artd tgn, ' . 

Pearson's correlation coefficient was^' computed for tha relationship, 
of ti^achlng exparience and the pi:oportion of test items at aach leyel, 
""■'Slmllai^y , the biserial correlatdon coefficient was computad and tested 
for significance to determine whether the t^icher variables. ^^ere related 
to the proportion of test items in each cognitive level. 



Findings 



The statistical results showed that no ralatlonship existed between 
professional inservice training and cognitive levels. A "moderate 
relationship" was 
and the level of t^st 
asking questions that r^equire comprehension and application. A "low 
correlation" existed between the teacher's stat^is and cognitive level 
questions* Full-^time teachers required somewl^t higher levels of cog- 
nition. 



^und between tha teacher's field of specialization 
t^st items j with those teachers trained ^n science 



The results of the ^alysis of variance indicated that no relation- 
ship "seem_9fc to exist" batween the level of the test item and the subfect 
matter taught. Similarly, there were no significant differences found 
for the lavels of, items generated by teachers of grade" seven in com-- 
parlson with teachers of grade ten. 

_Thc Pearson product moment correlation analysis of the relationship 
between teaching experience and /the proportion of items 'in each cbgni- 
tlve level showed a "moderate po|itive relationship" between tlie 
knowledge category and teaching exptL-riencG (reportedly significant at 
the 0.01 level) and a "low nci^ative relationship" when the correlation 
hetv^en teaching expariGnce and the items classified as Comprehension 
and Application. 
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'J^^ following is quated frDtn the article* - 

a. by far, thu hcnvl^'st emphasis in scivlnce exam Lnat: ions is on 
t^e lowast IcVl I of t:hc cians i f ica t ion sy s tem=-= the knowlGclge 
level. NGarJ.y -»72 percent of o^amination time is devoted to 
recall of f acts (30- percept of cliat time requires only' 
the lowest cognitive levels; | ^ 

b* only 7 percent of the examinat ion ' .^^ime is devoted to 
^ questions requiring the application*^ science principles ^ 

or other abstractions to' new situations' 

fc. test itt?TTi3 requiring comprehGnsion constitute 21 percent^.of 
(^^ ( the examinations; ' ^ ^ 

d, test itein.^ requiring the highest c^nitive levels, namely 
% \^nalysls, synthesis, and evaluat^ion are absent. 



tnterpre tat ions ^ ' ^ 

Bllleh concludes that the examination items prepared by the science 
temchers in Beirut are in the lowest subcategories of the knowledge 
category; there are no^ effective '^critical thinking" items. He attri- 
butes this to inadequate training in testing and evaluation and the 
Inability to identify important educational objectives. Bllleh suggests 
chat objective identification is a n^essary prerequisite to' the develop-- V 
ment of effective questions; su^h test items ^may be useful in the 
development of students^ cognitive processes, ^ 



ABSTRACTOR'S ANALYSIS 

Unlike many educational research efforts, the significance o^the 
results lies in the fact that significant differences between the 
various populations were not found. It would seem logical to 
hypothesize differing leve^ls of questions when comparing seventh and 
tenth grade teachers, better trained and lesser trained teachers, or 
the different science courses taught. Such differences were not 
found. Perhaps the most startling finding reported was that more 
items requiring low level s of cn^nltion seem to be asked by tearhers 
with the greater amount of toacliLnq experience. This would appear to 
be worthy of furtlier studv, ■ ^ 
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In general^ the stuiy. contributes to the body qf^knowledga centering on 
que^tionlngi^ objective formation, a^d evaluation. There are however, 
^^veral items in this report that must be discussed, some are minor 
^technlcaL^estions j others are somewhat broader In scope* ' - . 

Ali;hQUgh the author Indicates that the schools involved In the att^y wer^ 
ssle^te^d at rmdom, he dc^s not indicate how the apeclflc teach^s,^ 
C^aeses^ were arlected in those schodla. If ^25 schoqlsweye involved 
in the study and each school cbntained two classes, that would indicate ' 
thiit 50 teachers , would be Involved. Bllleh reports that the 33 examina- 
tions tised In tha study were submitted by teachers In 18 achooLs. J^at 
happened to the other aeven schools and 17 teachers? Was there som 
selact^^n that ray h&^s*^bl^sed i results? 

Another comppnent of the study that is not thorougjily discuased is thr 
proceaa of selection of the' Chrep^to-five hour sessions cohering the 
imit In each class. '^H^^^was the unit ^selected? Was there randomizations 
or an attempt to maintain similarity? Jt would seem that In soms. cases 
that to develop a one-hou^ exam covering ttaterlal that was learned In a 
three- to five-hour session may force the teacher' to over-emphasize low 
level" ep^ltive skills. As Bllleh points out, there was not even one 
question at the analysis qr synthesis level, ^ SoTChow, the design should 
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Insure that bath\the type^nd amoimt of ^ontent covered wtould be both of 
quality and quantity to provide the background nacessar^ for the develop- 
ing of higher copiitlve questions, Lf the three-hour class seaaion is 

devoted to low level cognitive learning, wKich may. be very appropriate^ 

■V ^ . ^ - / j/ ^ 

in the' context of the overall curriculum jj wuld the examination be 
any different In level? 

. • - ' s 

The establishmfftt of the alpha levels ranging from. 0,01 to 0,05 i^s *done 
so without any conmientary. It appears that the levels of statistical 
significance were used more for descriptive purposes than In terms of 
statistical inference. If calculations are being used for, descriptive 
purposes J then the level of si^iflcance is of less value than a discus- 
sion of Type 1 and Type 2 errors associated with the study. The 
researcher s commehts concerning Type 1 and Type 2 errors would be of 
as much, If^ not more, value to the reader than^^re the footnotes, -'Not 



significant at OrOS l«vel," or "significantly different from Eero at tire^ 
0,01 level*" Althoufji this abstractor recogpizes the problem faced by 
authors in getting research published that does not incliide sonm aspect?'' 
or tests of statistical significance, th^^e would be considerable ,bene= 
fit accrued to the rei^a^ch commimlty if some space and time were devoted 
to the items deecribed above. To be of maximum usej descriptive^ studies 
sust provide as much of the backgroiind data arid milieu of the investiga- 
tion as possible- 

The Investigator clearly limits the conclusions^ drawn from the study to 

science teachers in Lebanon, Since the sample was drara from that popu^ 

lation^ it seen© reasonable however, that there are' severaL ^hypotheses 

%^lch are Inherent in those conclusions that require further invest Iga^ 

tlon* Perhaps the most Important is that "sclertce teachers need to b^ 

trained in identifying important educational objectives in M specifid 

_ . J 
teaching setting." While this may seem logical, such a statement does 

not necessarily follow from the resuits pwvlded in this research paper. 

An equally valid hypothesis, recopitEed by Billeh, is that the real issue 

may lie in the teachers' inability to develop test items of a higher 

cognitive level. Perhaps tffift^ two issues go hand-in-hand, yet that 

is an assumption that must also be tested. 
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White, Richard T, and Lindsay Mackay. --D^rai. Bloom* s TamoomX^AEfiiX™ 
to Physics Examinations?-' The Australian Science Teachers Journal , 
18C^)l66-70, 1972. > v " - '~' 

Descriptors-^*^*Ach^ievOTiant Tests;^ Eyaluationi Evaluation 
. Criteria! ^Educational Researchi .-Physics; Science Education; 
. '*Sacondary School "Science; Test Cortstruction , 

Expanded Abstract and Analysis Prepared Especially for I,S,E, by Robert 
t, Stelner, The Ohio State Univerilty, ' / 



Purpose 




The researchers Investigated standardized examination item character^ 
Istics in terms of their possible importance in the preparation of 
valid examinationSi The Victorian^ (Victoria^ Australia) Matriculat^l^l^^ 
Physics Examinations of 1966, 19.67, 1968 were analyzed in terais of the ' 
subject matter content of the It^arts, tha item response fomat, and the 
cognitive level of the item ^ to see if these were relevant dimensions 
to ba considered in the construction of valid axaminatlons* 

Rationale 

The use of dlmansional grids *to assist in preparing course ax^inatlons 
is a common practice. It has generally been assumed that the validity 
of examinations is indeed Improved if items are selected according to 
cognitive ability level. It is expected ^that items with similar 
characteristics should be highly related, 

c ^ 

Research Design and Procedure 

This correlational study was based on an analysis of representative 
samples of studenc daca for tne Victorian ^Tatricuiation Fhysics 
ExaTninatlons of 1966, 1967 and 1968* Each of the Items contained in 
the examinations was classified on three dimensions. The dimensions 
werei 
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Subject Matter Content (3 divisions). Hhe examinatiQn itema^ ' 
were desigiied for the four sections of the PSSC physics text- * 



book (prior to the. 3rd 'edition) j^tth Sections I and III fprmitig 
one division, Sectionf II a seG^nd division and Section IV the 



third division. 



2. 



Item Response Format (2 divisions), nie items were either qf 
the completionior multiple choice response format and were, ^ 
^alus^fied accordingly. _ * ■ ' 

3. Cognitive Level (2 divistgn^^* ^ toowledge md comprehsnsion 

Items were classified in one division and all hl^er .cognitive 

* - * 

level items were grouped ttt;a second division. 

Each item was uniquely classified as fitting one of the twelve, possible 
cells of the three dimensions *' 

Subject Matter Content , . 
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Student responq^e to each Item was scored "1" if correct and "0" if Incor- 
rect. . Student score for each of the 12 cells was the sum of. correct 
responses to items classified in the particular cell. Twelve-point 
bisarial correlations were^ made for each of the items contained on the 
1966 , "1907 and 1968 examinations utilizing a representative* student 
sample for each examinatipn, > The 12-pQlnt blserial correlations for 
each item were made between studenCi score on individual items and student 
total score on each of the 12 cells. In the.,case^ of the correlation of 
the item to the cell to which it was classified, the total score did not 
Iniclude the item being correlatedp 

The correlation mat^x for each examination was used to carry out four 
separate analyses. In each of the analyses, the number of items correl- 
ating highest with cells of similar item Gharacteristlc dimensions as 
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compared to the nmnber expected dae to chancfi wag- used for statist^al 



analysis. 



In the first analysis, item correlatloris' with all 12 ceils were consid- 
ered. The frequency df the highes^^or^elation of items to cells to 
which they belonged was fxaMned*. " , 

In the afecond analysis j the frequency of the ^highest correlation of items 
^the 11 cells other than ..the one to whicH they'ti^longed was examined. 

In the third analysis ^ the frequency of the highest c5rrela£±Qn of the 
items to the eight cells of different content only ^s examined. This 
mm carried out for the 1966 examination only, % 

In a fourth analysiSs the frequency of the highest correlation of items 
to the four cells of different content and Item^ response format only was 
made for all three examinations. 

Findings 

In. the first analysis^ it was found for all examlnationB that the nAimbar 
of items which correlated highest with the cell* to which they were unique^ 
ly classified was significant Cp< *000l). 

In the second analysis * the cell to which the item was' uniquely classified 
was restricted from the analysis and the item correlations to the remain- 
ing 11 cells was eKaralned . The number of items in which the highest 
correlation was with a cell of similar content wasi" algnif leant for the 
1966 and 1967 examinations (p<.0001) and the 1968 )examinat ion (p<,01)/ 
.The number of items whose highest correlation wSs with cells of similar 
response format was also significant for the 1967 and 1968 examinations 
(p < t02). The number, of items with highest correlation with cells of 
similar cognitive level was not significant. 

In the ^hlrd analysis for the 1966 examination only, correlations of items 
with cells of different content only indicated ^that the number of items 
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whose ^Ighest Correlation was vlth cells of similar •Response format was 
significant (p < ,005) , ^ * = 

In the fourth analysis, which looked only at -item correlatiQns with cells 
of different content and response format j the number of items whose high- 
est correlation was with cells of similar cognitive level was not found 
to be significant. 



In te rp r eta t io n s ' ^ ' * * 

" ^ ' ' 

Based on the Results of the four analyses, the researchers CQncluded that 
both content and response format were relevant dimensions to consider in 
Iteias and ahould be considered in ensuring content validity, in Gradfe L2 
physics examinations, ^he researchers also concluded that there was no 
fcvldence from their analyses of the 1966^, 1967 and isks Matriculation 
Physics Examinations of the cognitive levels of classification as used in 
the study to indicate t^at it was a relevant dimension to be considered 
in relation to content Validity when examinations t^re constructed, 

y^ ABSTRACTOR'S ANALYSIS 

The methods use|i to investigate "'content validity in this s = =dy e differ^ 
ent^from those traditionally used in^evaluati^b studies , It not [ 
apparent that the invest igators have eKamlned^rontent validity , but 

r ' \ \ _ _ ' ^ 

rather the stMngth of relationships between Individual items and pofit 
hoc a priori dimansions of; physics examinations* 

The results are not particularly surprising and perhaps predictable. The 
dominant role of subject matter content has been shown in many studies 
and one would expect the highest correlation to exist here. The authors 
found that the frequency of correlation o£ items to the cells of similar 
cognitive level was not different from that duetto chance and concluded 
that cognitive level was not a significant dimension to consider in con- 
tent^ validity of physics examinat ions p The use of only two levels of 

■ 1 
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cognitive ability should improve the^acci|trfcy aijd reliability of \the item 
elasalf Ication, but there are still some inherent proBleM with.tfhe classi- 
flcation. Students we#^-GOB±ng from-.aumerpus back^^ of sctfools m 

teaeherSi What may have been higher cognitive level items fo^4ome may not 
have been so for others. The other dimensions of content and item response 
format do not suffer from the posslbljB overlap or mlxtog^f subgroups' ^as 
the cogniM-ve level classttica^lon does, 

The report is lacking somewhat in clarity^. This could be due in part to 
the brevity of the article, Tuller explanations of some aspects of the 
stfidy and the inclusion of some data and tables would have made the report ^ 
much more meaningful to the reader i / 

The authorSi^ indicate that a representative student sample was^'ttsei, but 
there Is no indication Off the^ample size, or the size of the studfent popu- 
lation from which the sample was selected j or how it was selected. The 
authors also indicate that som^ o£ the cells to whlcli the items were uniquely 
classified contained a smalt number of Items, but there ^s no Indication in 
the report as to how many items* A table indicating the number of items con^" 
talned in each cell for each examination wouli-hava been beneficial^ 

The frequency of highest it^ correlation to c^ls of similar Iten character's- 
istlc dimensions compared to that based on chance was used as a basis for 
conclusions* Although probability levels are given, there_ is no indication 
of the actual test sta^^stic used to determine significant results* 

The magnitude of the correlations is not given nor Indicated in the report. 
The analyses are based on the largest item-cell correlations, but there^ Is 
no way of knowing if the correlations are statistically or educationally 
significant. Since only the largest of the Item cell correlations was* used 
in- each analysis^ other significant ltem«cell correlations would not be con- 
sidered.- Again a table(s) for each of the four analyses actually giving the 

number of correlations and range in size with each dlraension (or each cell) 

- i % 

would have been useful for the reader to better understand the results. 



M^ny of. the above criticisms and suggestions result from the brevity of 

tl^e original article. It is quite likely that journal restrictions bcyohd 

I 
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the eontrol of the authors contributed to the omission of additional , data 
and tables which would have made the report much clearer for the reader. 
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EKpamded Abstracfr^and Analysis Prepared Eipetiially for I.S-f . by Richard M, 



Purpose ^ * 

Levenson and Brooks f#lt there was a need to Investigate the usefulness of 
student evaluations of laboratory courses because the goals and methods of 
laboratory teaching dif f eradvdrastically from those df^ classroom teaching. 
TOay ^elt the differences ^n milieus and purpose between the two tnstri^c- 
tlonal^ situations would n^essarily affect student p^rQeptions of their 
Instructors in each ca^ 5/ therefore^ there would be a dlfferentiA effect , 
^on student evaluations of Instructors depending upon the instruct^oiial 
v^ettlilg and purpose being considered. It follo^ed^ then, that taking Into^ 
account setting and purpose was of paramoimt importance when interpreting 
student ratings of faculty members. Further, the indigent state of knowledge 
concerning these, the setting and purpose variables suggested a strong need 
for additional research* In these regards, three primary research questions 
were confronted. ' 

1. Would students- evaliiations of their laboratory instructors differ 
significantly? 

2\ Would students' evaluations of their lecture instructors ^differ 
igslgnif Icantly? ^1 

3* Were there significant dif ferences betwyn studants ■ evaluations 

of their laboratory instructors and evaluations of chelr lecturers?^ 

• ■ • ■ ■ ■ ' ) . " 

Three working hypotheses were used as guidelines for the investigations X ^ 
They were: (1) students' evaluations would |ignif icantly differentiate among 
classroom lecturers; (2) students' evaluations would significantly differen= 
tiate among laboratory instructors* (3) students would rate their laboratory 
In^ructors more positively than thejlr lecturers. 



Rationale 

* 

While the investigations were hot cbnducted according to a specific models 
there were- two underlying assumptions which acted as points of departure 
for the research* The first assumption was that in the absence of sfensltlve 
field tested questionnaires, written specifically to, evaluate rtooratory 
Instruct ion j educational administrators wduld be forced to make defcislons 
concerning laboratory ^tudy based upon instruments whtehC^ere Invaald for 
that purpos%. Secondly, Interpreting evaluatiojis md making subsequent 
decisions without taking into account the setting might discburage good 
educat^lonal practices and penalize Instructors whose primary obligation is 
to conduct laboratories. 

Research Design and Procedures * 

The research reported herein was conducted at Texas ASM University and the 
University of Nebraska betwaan the spring semaster of, 1972 and the spring 
semester of 1974* It Includas one main and two. corollary studies. The main 
study and one corollary study were conducted at Texas A & M University while 
the second corollary study was conducted at Nebraska University * All sub- 
jects participating in the studies were first-year chemistry students. 

/ 

Samples I ^ ' ' 

There were three . samples in the investigationi (1) the main study involved 
a sample of 329 students Randomly selected from a first-year chemistry popu- 
lation of 1,600, This sample was not controlled for sex; . however, the authors 
do Indicate, the inclusion of insufficient numbers of females in the sample to 
allow for the control of the sax variable; (2) the first corollary study 
Involved a sample of 2,00 Texas ASM students; (3) the second corc^lary study 
Involved a sample of 19^^ University of Nebraska students. 



/ 



M0thods 



The methods used in* each' study are described by Individual study: (1) ^The 

main study evaluated professors who Xpent three hours each week lecturing 

to students and a short pewod of time supervising prelaboratory instruction, 



and graduate students who spent about three hours each week conducting labor- 
atory sessions but did not lecture formally to] the stndents. During the 
laboratory se*ssions the only Instruction provided by the laboratory Instruc-- 
Cp^JF^yas on a tutorial basts. Following the completion of the spring 1972 
semester the subjects were asked fio complete two questionnaires (deigned 
by the InwstigatorH) which conrtined items related to most-liked and items 
rela\€d t^^lemst-liked teaching techn^ues. One questionnaire was intended 
for evmluatioti of the lecturer while tHi^ second dealt with evaluation ofi, the 
laboratory instructor. .. ^ 

After completion, the questionnaires were content analyzed for both most- and 
least-appreciated teaching techniques on a 1 to 3 point system. ^ I^the former 
category a score of 1 on an item indicated a high degree of detail and Appre-^ 
^iation^while a score of 3 indicated little detail and appreciation. The 
latter category was scored conversely. A score of. 3 on an item represented 
a highly detailed and highly unfavorable rating and score of '1 represented 
a favorable rating for that trait but the response lacked detail. Eleven 
lecturers and ten^ laboratory instructors were evaluated in thii manner. 

(2) 1 The first corollary study asked subjects to describe^ ^in detailed writ- 
ing, the conduct of the last lec and laboratory session they attended. 
The scoring of this .instrument was based upon word count techniques. 

(3) The second corollary study was conducted during the spring 19^4 semester 
and used the same Instrument and evaluation scheme as was used inl the main 
study. In this study studertts evaluated graduate student lecturers who also ^ 
were their laboratory Instructors. . The graduate students spent one hour 
each week as formal recitation instructors and three hours each wc^ as 
laboratory instructors. They were evaluated as lecturers and as lab instruc- 
tors in the same manner^ as were lecturers and lab instructors In the main study. 
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Data An alysis ^.^^ 

All hypotheses were evaluated via analysis of variance techniques and the 
maximum probability for making a Type I error was set at P < .01. 



Flndlnga 



The findings are smnmrlzed In two sections. One iumps the findings of the 
main sti^y and the corollary study conducted at the University of Nebraska 
and the second Includes the findings of the corollary study conducted at 
TeKas A $ M University, ' ■ ^ ^ 



Main Study and Nebraska Corotlary Study ■ 

. • ; 

Although the tnaln etuHy involved both senior faculty members and graduate 
students while the Nebraska corollary study Invol'ved only graduate students^ 
the results of both Investigations were essentially the same. They were: 
(1) questionnaire items concerned with most ^appreciated teaching techniques 
failed to differentiate si^ificantly among lecturers, among laboratory 
Instructors or between lecturers as a group and laboratory Instructors as 
a group;, (2) questionnaire items concerned with least-appreciated teaching 
techniques differentiated significantly among lecturers ^ among laboratory 
instructors at the ,05 but not* the *01 level, and between lecturers as a 
group and laboratory instructors as a group* In the case of the third find- 
ing| subjetts were more critical of lecturers tnan of lab instructors; (3) 
the most positively rated lecturer was on the positive side of the mean item 
scores for both most- and least-appreciated teaching techniques and the most 
negatively rated instructor was on the negative side of the mem for both 
types of if ems. 

. .. ^ ^ A ^ . ' 



Te3oa3 A & M CprcDllary Study 



The following results were obtained from the data analysis in this study i 
(1) a total word count failed to differentiate significantly between lecture 
and laboratory settings; (2) subjects wrote more about the first type setting 
they evaluated* (3) subjects wrote more about lectures if the lecturers were 
not present and th^- Ss had just completed a lab session; (4) subjects wrote 
more about labor^ory settings if the lab instructors were not present and 
the Ss had just finished attending a lecture; (5) when numbers of first 
person singular and plural pronotms used were analyzed there was a highly 
significant difference between the laboratory setting and the lecture setting 

■ ■ , 38 , 



favoring the lab setting; (6) when lectures were dpsdribed firstpjtwice aa 
many first person singular and plural pronoims were used than when leatures 
wer^ described last. 



Intarpretatlons 



.^Jaa^atjgafce^a. dgeHJthB^jQ^ f Indings based 

* upon the use of an' open ended or subjective questionnaire format, 

1, Least'-appreciated teaching technique related questionnaire Items 
can meaningfully discriiitnate between lecturers* 

2, Students are more aritlcal In their evaluations of lecturers 
than they are of laboratory instructors* 

3, Most^appreciated teaching technique related questionnaire items 

^ are. not good disc^minators of lecturers and laboratory lastruc^ 
tors, 

4* Students preceive lecture ahd laboratory settings differently* 

_ _ 

5, Instruments which are effective in evaluating one educational 

setting are-^ess effective in making evaluations in another ^ ^ 
setting *^'^ . . 



ABSTRACTOR'S AI^ALYSIS 

f 

In this period of Increasing detnand on the part of taxpayers and students 
for faculty accountability, the burden of reGoncilJ.ation of the issue has 
fallen upon the educational administrator, Whp^^will be tenured and who wil4" 
not, which programs survive and which meet their demise and many other diffi- 
cult questions are related to good and ongoing faculty evaluation programs* 
They are questions for which there is no simple answer and they are questions 
with which even the most knowledgeable of administrators finds difficult. It 
follows that those unskilled in the art of faculty evaluation may sprve only 
to frustrate able faculty members, discourage good programs and generally 
create havoc amongst stable and good edu^tional environments* 

39 . 



Since inAny educational administrators enjoy expertise in areas other than 
faculty evaluation instriiRc'ftt development^ testing and validation, the bur-^ 
den of such activitia.^ r'nXls upon those ho schooled. It is to these concerns 
and to an obvious void in the li terat ure that the authors h^va addressed 
0^ their research. 

The concern that instruments developed for the evaluation of one educational 
setting inight not have validity when applied in a different setting is valid. 
Further, the validity was supported by the results of *the study. 

The results of this study suggest a need to be extremely cautious when inter- 
preting the results of any faculty evaluation* If the evaluator is not well 
versed in interpretation and/or familiar with events surrounding an evalua- 
tion instrument's development ^ then advice and counsel should be sought from 
colleagues and other professionals who are more knowledgeable in the area* 
I 

The results of t-lle study suggest that certain types of evaluation items are 
viable discriminators of weak and strong faculty members while others are 
not. However, this conceptual contribution must be used with caution since 
it is based entirely upon student perceptions. To generalize to a inore 
global population at this time would be premature. It might be that , if 
the research were duplicated at different- institutions , controlled for age 
or sex, the outcome would be somewhat different. 

Research design also has a strong influence upon the outcome of a piece of 
research. It could be hypothesized that different results would surface if 
graduate student lecturers were evaluated based upon the same number of 
student -lecturer contact hours per week as tho f aculty-lecturers , Another 
hypothesis suggests that if institutional goals and objectives were con- 
trolled, the outcome might favor other conclusions. Random sampling for 
all studies, while perhaps not always essential, always strengthens the 
validity of conclusions. Generally speaking, wlfen one considers the possi- 
bility of institutional constraints^ the problem of obtaining willing 
subjects and the myriad of other confounding variables^ only one conclusion 
can be made concerning tlie research design; that Is that the design was a 
good one. 
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PerhapB the most scathing criticisms made of contGrnporary research in any 
field are very subtle* Those for whom rcscarcii would bo of most benefit 
quietly refuse or openly resist reading tlie, latest journal articles. It is 
not until researcli is re --writ ten and incorporated in various compendia and 
othar textual materials that many educators become aware of i-ts existence 
and by this time it is old and perhaps dated, Thd question why such events 
cowB to pass is not an easy one to answer. 

Placing the reading and subsequent use of contemporary research within its 
proper context, it must be remembered that the majority of research is pub- 
lished as joiirnal articles* Journal editors today are being confronted with 
periodic price increases which seem geoTnetric, Since production costs equate 
to document and individual article length, editors and editorial advisory 
boards encourage investigators to force the most information into ^Jie small- 
est space possible. The concomitant results of these events lead to the 
production of articles understood by only those well versed in the particular 
subject area. 

With these comments in mind, several criticisms and suggestions are advanced 
which might make this excellent piece of work easier to follow: 

li The use of the phrase "is to make in and between comparisons of 
the ratings students give to their lecturers and laboratory instructors 
In introductory chemistry" is vague. Many readers would find the use of 
"In'* and '"between" difficult to understand and become frustrated early on - 
in their reading, A more detailed Cnot verbose) description of the purpose 
of the paper would aid the reader greatly. 

2, In the case of the main stud v. it is difficult to tell whether the 
lecturers were being evaluated based upon three hours of lecture each week 
or upon the s rvision of the pre laboratory instruction, A need for clari= 
fication exis nere , 

3, Tlie open=ended evaluation quest ionna ire would be easier to under = ' 
stand if two sample items were included, one related to most-- and the other 
rtjjated to least=^apprec Lated teach inc^ technLqunH, 

«f*- 

4, The reader would be well Herved If tlie fnct th_at there was a main 

and two corollary studies liad been mentioned at the beginning of the 
"Methods" section of the papn'. 



5, One must question how the subjects were chosen for the two corol- 
lary studies. Also, wore the 200 subjects in the Texas A St M corollary study 
some of the same subjecty who participated in the niain Htudy ^and/ur were 
they dra^vTi from the samG chemistry population ay tlie subjects in the nmin 
study? 

ff^' Did the TeKas A & M corollary study evaluate the same lecturers and 
laboratory instructors as did the main study? 

7. The results of the main study are located in the "Results'- section 
while those of the two corollary studies are . found in the "D,iscussion" sec- 
tion, , The paper would be easier to' follow if all of ' the research results had 
been included in the ''Results" section* 

Aa was previously mentioned, the state of the art concerning the use^ of eval- 
uation Instruments across educational settings can only be described Us 
indigent. However, the results of this research point to a need for addi- 
tional work in the areaj especially if educational administrators/ faculty 
development officers ^ anti others are to use student ratings to best advantage 
Therefore, the following suggestions are made for additional research, 

\ 

1, The sex variable should be investigated, Spa^lf ically ^ does the sax 
of the student differentially effect the outcome of the rating, does 
the sex of the lecturer or laboratory instructor differentially 
effect the outcome of t^ie evaluation, arid are there any sex^^sex 
Interactions which confound results when sex is used as a variable? 

2, Since this resoarc^i suggests a diffarence in the relative abilities 
of certain instrumerits ^ to evaluate different educational settings, 
"there Is a need to continue development of evaluation instruments 
for specific educational settings. 

3, The question of whether subject age is a factor in the way an 
instrument differentiates within educational settings should be 
Investigated, 

4* The Question of whether an evaluation instrument dove loped for use 
in one labo rat c? ry/" !-;ub j (n:t afon can also bv used In , other ,lahorrrtory 
subject areas needH to bo nnsworod , 

/ 
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5. .Finally, the question of whether or not this study is duplicable, 
should and must be answered* it should be rcmGmbered that the 
generalizab ility of a study * s results depends upon the results of 
^similar and idont ical stud ies , To generalize based ugon the ^results 
^ of one study courts Gon*fuHion and the propagation of half-truths* In 
this regard, one might question what effect the study's designers 
had upon the outcome of the study* 
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Purpose 



The major purpose outlined by the. author In this study was to identify 
^erroneous notions (science misconceptions) about soTne scientific facts 
and concepts and to deterTnlne the extent to which they are prevalent ^ 
among Certain groups of students in the Beirut, Lebanon area, The study 
also aimed at determining if ^proneness to science miscQjicept ions is 
related to the variables of years -a^f^^ education, sex, major in sciencej 
achievement in science, and culture * 



Rationale 

One of the considerations in effective science teaching, suggests 

- _ * 

Za-rourj is the identification of m=lsconcentions in the popular notions 

of scientific facts and concepts, Za'rour reports that Weaver (19^5) 
conducted a survey of physics misconceptions present in twelve series 
of science textbooks for elemGntary schools and seems to have found 
none frfcc of' misconceptions. The author also reports that Garone (1960) 
pointed out that science misconceptions could be traced to improper 
reliance on common sense and to the misinterpretation of one's experi- 
ences* Za-rour suggests that if misconceptions are related to irrational 
thinking or to a misinterpretntion of the cause^-and^ef f ect relation, lip 
as explained by Hancock (19A0) ^ tiien proper t rtf'c h in g-- learning situations 
aimed, ^at fighting these shortcomings should help in reducing misconcep-- 
tlons. Za'rour reports that Kuetho (1963) has shown that the awareness 
of the prevalonco of science m LHConccp t i ons on tiic part of the teacher 
can direct teacliLng toward a clear differentiation of a concept from other 
concepts that have a htgh pro!)ahnity of intrusion, 



The subjects were higli school freshmen and Juniors from 11 high schools and 
practically . all university sophomores at tlie American Vnij^c^&ity of Beirut 
(A.U.B*)* The combined total was 1,444 students/ Except 'for 130 American 
students from the American Community School (A,C.5,), ail the studGnts x-jere, 
or had been befor^ joining the university, part of the Lebanese System of 
education. All students in the sample were from a wide variety of^ schools 
'in Beirut -and its surroundings* University students who'' did not study through 
the Lebanese system of education in their high schools were excluded, * 

The process of developing the test was perform.ed through a teview of pre- 
vlousystudles 5 interviews with teacherSi and dra%^in| from the authorVs teach- 
ing experience. Two tesr.s with a total of, 64 i\ams were piloted* These 
consisted of multiples-choice, true-false^ and open'-ended items. The idea of 
a multiple-choice item was retained If^ it had a distractor whldh was more 
.attractive to the pilot-^study studehts than was the correct an;swer, The^ 
ideas of true^false and open-ended questions were generally retained if the 
majority answered ^them incorrectly* Dls tractors which were not attractive 
at all were rep^ced and erroneous responses to the open^-ended questions 
were transformed into dlstractors of multiple^choice questionSi^> A new 
version of the tests was tried to check its language and construction. Final 

i 

modification resulted in a 40- it em multiple choice test with four alternatives 
per item* AboiTt-ZO items were in the physics area while the other 20 were 
distributed among earth and space science ^ chemistry, and biology. The 120 
distractors include 12 of the "none of these'* an^ "impossible to tell" type. 
If these were set aside, 108 erroneous ) science statements or |5otential mis-- 

conceptions were left* - . , 

\ 

The test was a dm Inisttired to the studerits in their respect i v schools or 

i _ 

classes by the researcher or an assistant. The percentage popularity of t 
each alternative was computed for the total sample and fo^ each of the 
different subgroups wliich were involved in the va^ lab let to be studied* 
The responses of the 130 American students from wore analysed^ 

\ 

separately tljiroughout the studv* A distractor which was selected by a 
percentnp^e ^renter (nt the .05 Icvol of s Ir^nl f icnnco) than the expected 
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chance scorft was labaled a misconception, t^-tests were used to test f.or 
significance, A Pearson r was calculated becwecn the test score of 
correct responses on the ml sconcep t ion test and each of the following i l-* 
science^ grades of the stiidents from two high schools^ and the verbal 
and quantitative scores on the Scholastic Aptitude Test (SAT) of the 
eleventh graders from A,C,S, "'^ "~ '^^^ 



^ 



The results indicated that 20 out of the 108 diatractors or potential mi's- 
conceptions ware selected by 30 percent or Tnore of the respondents. Tables 
In the report list the ^misconcept ions and their average popularity and per-* 
centages at the different cla^s levels* Considering the significance of 
difference in percentages between the ninth grada and the university group 
in these itemSg it was found that there was a significant decrease in nine 
cases, an increase'^ in "one case (the weight of an object at the North Pole 
when compared to its weight at the equator is smallar)^ and no significant 
change in the remaining ten items. 

In studying sex differences, there appeared to be a differenca in favor of 

the males in the total number of misconceptions held at the elaventh-Erade 

^^ i _ " " """""" - 

level* The differences betwean niales and females at the ninth grade and 
university level was mue^ j^ess pronounced* 



Comparing the students of the A.C^Sp schwl ^ith comparable Lebanese stu*- 
dents did not reveal appreciable differences. There' was j,a qualitative 
difference j however ^ in that there were dif f erenr-items misconceptualized 
by each group. Students of these two schools did significantly better 
than schools enrofling students from a lower socioeconomic class * 

The results reflected a significant improvement in performance of those 
iStudents in a university x^ho had or were completing two to four science 
semester courses at the university when compared to those with no or one 
sclGnce course. For the group of /Xmorican eleventh grade students who 
had In their records the SAT scores, the correlation coefficient between 
the scores, of correct responses on die misconception test and each of the 
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verbal and quantitative SAT part scores were ,41 and ,68., ruspec t ivelv^ 
At 'the two scliools where science grades were obtained, the correlation 
coefficient between the scorfes of corrGct responses and science grades 
were ,51 in one school and A7 in the'other? 



Interpretat lon_5^ 

A significant steady decrease in adherence to misconceptions with Intrease 
In education occurred only In two items which are included in the science 
curriculum abanese schools ^ while 4 of the 20 d terns shown to be insens- . 

itive to level of education are not directly taught as part of the curric- 
ulmn. One item (the most misconceived ty most students), "l^en compared to 
fflolst air, 'the density of dry air is smaller*- seams to be due, the author 



states, to a misconceived cormnon-sense notion that wet objects weigh more 
than when they are dry and this is then generalized by students to dry and 
moist air, , 

m 

The^ results of this study showed that the females held significantly mora 
misconceptions (11 out of tha 20) than did the males at the eleventh grade, 
but there was little difference between ninth grade girls and boys and 
un|versity men and women. This appears to ba somewhat at variance with 
^findings of Baily Ci962) who reported an overall tendency for boya to hold 
^ fewer misconceptions than girls and lidler (1966) who found that men college 
students surj>assed wo^n students in the undarstandlng oS science concepts, 

- - - ) 

The qualitative differences, the author states, between the performance of 
the American students and those of the Lebanese students of cbmparable 
status may be due to cultural differences at home and/or different methods 
of teaching and curricula. Considerably more Araer ■ students than com- 
parable Lebanese students have tlie inisconceptlon that air is mostly composed 
of oxygen. The author raises the Question* Do the American students speak 
more of the ^foccurrence of oxygen or r4^^e local students dr^rlled on remember- 
ing facts about percentages of gases A th^ air or is it a combination of 
these two factors? 



The finding that fewer niis^nceptions are held by those who were complet'- 
Ing more science courses at tli&^unlvcrslty is in agreement with the resitlts 
of Adler (1966) and is in disagreement with Boyd (1966), 

The study, the author hopes, will motivate teacliers to analyze the Incor- 
rect responses of students on test questions because of the valuable 
feedback that it can provide and in t^_e assistance it can give to the 
teachings-learning situation, M 



ABSTRACTOR^ ANALYSIS 

This piece of research suggests to teachers that they not only look at 
the right answers that are given by individuals and/or groups in examina- 
tions but also be particularly alert to those questions where a significant 
number of students give a wrong answer to a question. The teacher, then, 
can counteract the misconception by providing the proper curricular mater- 
ial or experience to correct the misconcept i^n , 

The use of open-ended questions and tlien studying the responses is neces- 
sary and good but a t ime-consum_ing method in discovering the inisconcep t ions 
students have. The researcher and science teacher need to carefully examine 
responses on tests and then lump similar answers in determining 'com_mon 
errors. The pay-off is that these can be cdnverted into more ■ obj ect ive 
(such as mult iple-^choice) questions which then can be used to quickly 
survey a number of areas and to alert teaches and researchers to problem 
areas. In summary^ the design of the test employed in this study seems to 
be a fruitful way approach the discovery of students' misconceptions, 

The misconceptions in the test reported in this research vary from fact 
(e.g,, "AJir is mostly composed of oxygen'' and "Bones make up most of the 
weight of the human body") , to concepts which are basic to certain under-- 
standings in science (e,g., Vt'Then compared to the work that is put into it, 
the work can be obtained from a simple machine is^ usually greater" and 
"Ships float on water ^because they are made of material less dense than 
water"). It Is more impnrj.iint, obviouHly, to identify misconceptions 
having to do with conc©^ts than those having to do with facts. For the 
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most part In this study the Tnlsconcep tions deal with concepts rather than 
merely facts. 

It would be well to identify those misconaap^tions which could be remedied 
by demonstrations or, better stilly by hands-on exper Imantation , Unfor- 
tunately some of the misconceptions reported in this study are associated 
with abstract models [e*gt, "As you're listening to a radio using electric 
Ity at home^ the electrons that flow into the radio all change into energy 
(light, sounds ot heat)" and "To change an element such as nitrogen into 
another element such as oxygen is impossible,"] and no hands-on experiment 
tatlons can be provided. It is questionable whether eKposltory teaching 
will easily clear up these types of misconceptions. 

The test wai composed mostly of concepts from physics. It would be useful 
to Identify the misconceptions that occur in chemistry, biologyj and earth 
and space science, Coimnon misconceptions in biology sjch as plant cells 
have cell walls whereas animal cells have cell membranes or plants photo^ 
synthesize whereas animals respire are two connnon biological misconcep-- 
tlons, for example, that the abstractor has experienced in his teaching. 
If yr^ could identify more of these-^ypes of misconceptions made by ^' 
students, we then tnight be able to design curricular materials to Include 
sufficient experimentation and material to help alleviate problem areas* 

It would be interesting to try Mr, Za'rojbr's inisconception test in Ameri- 
can settings at schools varying in socioeconomic levels to find out| how 
widespread and how different are the misconceptions among various groups, 
Also,^ cross-cultural studies might reveal under close analysis why it Is 
that students in some cultures experience ease in niastering concepts and 
why large numbers of students in other cultures are left with erroneous 
notlonsl^ ' > 

Also looking at girls and boys in various cultures to see how they differ 
may give clues to why boys do better than girls in science tests in some 
societies and not in others. It is not just important to see that there 
are d 1 f f c r en c cs bq^^f we e n g 1 r I s and b oy s on who 1 c s c 1 c nc e tests. It is 
most valuable to know on what Items these diffarences take place. Since\^ 
the test has more phystcnl science items than items from the life sciences, 
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It is tjurprlsing to the abBtrnctor that sif^nif icant diffcrances botwcGn 
girls and boys did not occur at the ninth grade and univcu'sity- levels. 
It may be that, in thej Lebanese society , different cultural forces are 
operating f rom tiiose in the American cul^ture* It would be well for 
American researchers to compare differences in boys and girls and men and 
women on misconception tests in the life sciences; it may be that males 
and females are on a more equitable footing in the life sciences and less 
SO in the physical sciences ^ aga^n because of cultural differences. 

The National Assessment resultsf in the United States and other standard- 
ized science tests can provide aMdi^tional items where large numbers of 
students have faulty notions in^ science. The National Assessment results 
can be used to identify misconceptions of 9^ 13 and 17 year old American^ 
udentSj and these misconceptions added to Mr. Za'rour^s list. For ^) 
instance, for the 13 year old, misconceptions such as the following can 
be identified by looking at popular choices of the distrac^ors on the 
test* "When a parson faints, one should lay him down and apply cold 
packs" and* "Ice tnelting most clearly forms molecules different from, those 
present at the start" and further, ''A- fossil of an ocean fish found in a 
rock outcrop on a mountain probably means the fossil fish was pBpbably 
carried to the mountain by a great /flooc 

Also, it is possible to identify, by looking at the/results of this test, 
what misconceptions are held bf' l? year olds. These, then, can be used 
to add to Mr. Za'rour's list. In addition to the identifying misconcep- 
tions in fundamental facts and principles of science, the National 
Assessment instrument instrument will identify misconceptions about 
abilities and skills needed to engage in the processes of science and 
the Investigative nature of science. Not only is it important to identify 
misconceptions having to do with the product element of science but the 
process element in science as well. 

But even after identification. of a ^asc number of such misconceptions in 
the various scientific disciplines, there remains the difficult task of 
deciding on the means by wlT'i^h to correct thpHC Tiilsconcep t inns in students 
minds. Is it enouf?,h to point out the error, give the correct answer and 
hope the student then assinilates tlic inlooiation? Obviously this will 



work with some studynts, but it may be that in order to correct some basic 
misconceptions for a great many students, e^fenslve teaching has to be done. 
Certainly this is true* Lor Home of the more basic and/ difficult concepts 
embodied in some of t.he misconceptions . 
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1974. / ' ^ " " " 

Descriptors- = '-Curriculu^N^cvGlonment • Tndivicfualizod Frngrama i 
^Ins truccional Des LV^n; Junior llif^h Schoolri ; Program Descrin^ 
tions| ^Scienca Curriculum; Eicience Education; Science 
Materials I '^Science Programs; '^Secondary School Science 

Expanded Abstj^ct and Analysis Prepared Especially for I.S.E. by Robert 
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\ . 

Purpose 

> 

The major goal of the report is to describe the goals, rationale, and 
procedures which were used in developinp, the Intermediate Science Curri- 
culum Study (ISCS) which was initiated in 1961. The story of the plannin 
and development\^f ISCS is reported for instructional designers who are 
concerned with curriculum innovation and student imnact. 



Rationale 



L 



Previous attempts W itidividualize instruction are mentioned as fairly 
conmon occurrences. The ISCS authors began in 1964 with investigations 
as to why past attempts resulted in little more than local impact. They 
found that'^ all wore the work of few persons with ideas and nearly ideal 
teaching conditions* All tended to fail when they were transported 
generally to the real world. This real world involves considering such 
general issues as school budge ts, school design, teacher preparation, 
parental expectations, and state rec^ulations , 

The ISCS developers sought means for considering these problems while 
proceedirig with a practical model for individualizing science for the 
Junior high school years. They argued that solving such problems and 
developing such a model would tal^e considerable time, resources, money, 
and full attention of t op = level pe^onle. They were able to secure a 
$1,600,000 grant from the Office of Education and l^ater a supplemental 
grant of ^3,500,000 from the Matlonnl Science Foundation. With such 
resources ,)^-^t'-iirio_Ws availah as well as facilities and talented people. 

f 
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More than ^50 pcr^on^R wera involved with thn efforr, prnvi(Iin<^ a m:isRive 
flood of ideas and tha InvolvcmGnt niirsons wlio would he a f fee tad hv 
the pro ^^r am 

4 ' ' ' 



i! 



^ ^^^earch Pes ^fTn^anj Proce dure 

Initial efforts inclwdeM estimates concerning the degree of innovation 



s 



that junior high schools could and would tolerate. It was found that 
teachers were shal lowly and narrowly Drepared in science* Many were 
noTi-majors. The style of teaching was didactic with few activities, 
especially .those that could be called problem-sol vingp An early decision 
was to build the desired content and Instructional rationale into the 
materials* to enable?- teachers to use them js^ith little or no special train-= 

A program for use in rooms with poor laboratory facilities and resources 
"was planned, Low-^cost kits were designed which contained, all materials 
needed* Anything requiring special facilities in the classroom or expen^ 
sive equipment was eliminated. Printed materials were selected as the 

rimary vehicle for cominunication with students because of state textbook 
regulations 5 teacher skills with media and tradition* 



Findings 

— . ) . ^ 

J 

Two types of instructional materials were produced* Core mte rials deal 

with concepts that are important for all students. Optional activities^ 

A - 

are called excursions and are of two kinds. One kind of excursion pro- 
vides for remedial work while another provides interesting extensions for 
motivated students* The materials include about equal amounts of core 
ccBi^ion materials. 



The materials assume that nil students will complete all of the core 
materials bu^^^t a pace the student elects. Students encounter "check- 
ups" within the materials that may direct students to remedial activities 
if needed, ITie materials include roferencoH tci.othGr excursLons wliich 




moTm qllwtltatl^^ consideration of^ a concept « 

Thm oaterlals purport to utilize the Plagetian model for intellectual 



tranaitions which occur "In the junior high school ,in a nu^er of ways- 
Most activities require students to handle concrete objects. The more 
abstract concepts are delayed to the end of the sequence when students 
can better handle some abstractionp Disequllibration is facilitated by 
continually putting students Into problematic situations for which 

r 

rational explanations are sou^t* Choice of concepts that are included 
are based upon whether they are needed and will be reinforced by subse- 
quent use* Questions are Included frequently as a means for increasing 
student motivation and involvement* 

Interpretations ^ - 

The author describes the ISCS approach as a "semi-systems" one* The usual 
products of a systems approach are specific Instructional objectives ^ 
instructional materials directed at accomplishing the objectives , and 
evaluation materials to determine when objectives are met. The procedure 
actually used, however^ Is termed "semi-systems" "since the objectives were 
formulated after the develppment of instructional materials. Thoj ISCS 
group found that the writing group was improductive as they grappled with 
stating specific objectives* IJhen the decision was made to abandon the 
effort* there was a burst of productivity and a volume of "good" instruc- 
tional materials resulted* The developers state that scientists often 
believe that most really important objectives of science instruction can 
never be stated concretely. The ISCS developers concurred with this 
rationale since scientists were needed and they could not work produc- 
tively when asked to prestate instructional objectives. The "semisystems" 
approach resulted in quick production and school trial. 

ITie activities which op©=tfPced between June 1967 and May 1960 were selected 
as typical for dtscj^iting the ISCS developmental effort* This included 
a seventh grade revision and a first draft eighth grade program* About 



development* Th^| develon^s seek to emphasize the usual psychological 




40 aclantlsts, teachers and "educationists," si^ported by artists and 
edltoifai were responsible for the written materials, equipment kltSj and 
evaluative Instruments for 10,000 students to use during the 1967-68 
school year. 

nie project staff processed seventh grade feedback Infonnatlon from the 
field trial of the preceding year* They also outlined the planned eighth 
year a^uTsm^ The summer writing teams were then assigned specific tasks 
with rather rigid tlTO-linea, Each seventh grade team was assigned a 
spCTiflc portion of existing materials to be revised. All seventh grade 
materials were revised within three weeks. Each ei^th grade team was 
given one week to draft one subject with a rou^ draft of the entire 
course completed in ten days. Reconstituted teams then produced a second 
draft with another one^week deadline. The proces*s was repeated several 
tinss with all parts of the program imdergolng least three drafts. 

The authors prepared prototypes of all equipment. Suppliers were con- 
tracted to assemble the actual apparatus kits. By mld--September all kits 
and trial editions were in schools In five test centers for field trial. 
In addition to the actual field' tea ting* a group at Florida State ^exper- 
lanced the materials via a conrputer assisted program. This provided 
another important vehicle for evaluating the program. 

As the evaluation Informtlon was collected, formal performance objectives 
were committed to paper. They were drafted by project staff members. 
Self-tests were developed quickly after the objectives were formulated. 
Although large numbers were utilized for the sumner writing conferences, 
the final editions involved relatively few persons. The ISCS experience 
suggests that many persons are good for mass productions, for ideas , for 
inventions. However, relatively sniall numbers are best for refinement and 
preparation of a final version for publication and widespread use. The 
ISCS program is described as an innovative one utilizing an individual 
approach. It has had major impact upon jimior high school classrooms. \ 
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ABSTOACTOR'S ANALYSIS 



"An Approach to instructional Design for Massive Glassroom Impact" Is an 
t&tareetlng account of the ISCS story as viewed and reported by the Ini- 
tial director of the project* This kitid of analysis Is Importartt for the 
research coBmmlty. In many respects It Is the kind of observational 
raporb for which many researchers yeam. Yet it is not a research report 
in any tradltimal sense* ^ ^ 

The raport is a brief accowic similar to the Grobman book describing 
the Biological Sciences Curriculum Study (BSCS) effort (1970) and Karplus 
and Thier's description of the Science Curriculum Improvenient Study (SCIS) 
project (1968), Both of these accoimts, hpweverj are mDre inclusive and 
provide far greater documentation. 

Relating the ISCS effort to other attempts at individualizing instruction 
is both Interesting and valuable* Similarly, terming the effort a "semi= 
systeii^*approach" provides a valuable framework for assessment and compar-- 
Ison* The analysis of Piaget's model and the ISCS program is also of 
general Interest. It would be of greater interest for the author or ^ 
others Involved with ISCS to develop any one or all of these strands into 
a comprehensive analysis* In one sense this article merely scratches the 
surface and does not establish the assertions as such in the program* 

The author does not approach science curriculum theory and the part ISCS 
might play. Certainly there are major differences both in the materials , 
the times j and schools from the national curriculum efforts in science 
pre-ISCS and post-ISCS, At tilths the author seems less intent upon 
reporting specific approaches that were used several years earlier as 
opposed to emphasizing procedures and directions for 1974 and beyond* 

All too little can be included in a seven-page Tnanuscrlpt of this kind* 
However^ observational information is valuable and should be collected. 
It would be of Interest to compare this^l974 report of the "approach" to 
ISCS desl^ to reports released in 1967 and 1968, It would also be val- 
uable to reduce this report of Instructional deslpi. to its slmplist 
procedures and to compare It to other national effortJtboth in science 
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and othMt fields. Further, it would be worth cdmpfl^ring a mmtm eon^lete 
description of the ISCS development with the reports of Gro^tfen on BSCS, 
Karplus and Thlar SCIS, and other directors of cuVrent national 
eurrleultm efforts. This could give us an Interestijig 20-3^ar** history 
. Qoncerning one of the most extensive curriculum developmant perlpds wer* 
/ ' - \ ^ 

'^An Approach to Instructional Desl^ for^ Massive Classrootfl Impac^' was 
well^wrltten and of general value. Because of its brevity * however * 
there were many assertions, arguments, stateMnts of fact that create 
manswered questions* Some of these djnclude the fpllpwing: 

What factors (features) of ISCS design make it "innovative"? What is 
the history of individualization? How does pacing of prograBmed materials 
result in Individualization? Which other "Individualized" programs were 
developed with the view of general use? Where is the specific information 
that describes the 1966 science teachers. Junior high schools, th^ exist- 
ing materials? What is the average cost for jtffilor high science? Pre- 
ISCS? Currently? With the Information presented (including rationale 
concerning content and approach for both teachers and students) what is 
the explanation for the need of special teacher preparation materials? 
Wha^g^ some of the specific outcomes of ISCS Instruction? ^^at are the 
objectiws and to what degree have they been met? Is "impact** measured 
raly In terms of numbers of studants? What Is the source of the figures 
regarding students (from publishers)? What Is the "experience that has 
bottle out that Interest has tended to remain high-'? Is there specific 
information concerning student motivation with ISCS materials. How were 
initial staff objectives for materials (and for students) different from 
those formulated after field testing some ISCS materials? Since objec- 
tives were staff prepared anyi^ay^ is it valid to report that "scientists 
found it difficult to prestate objectives'^jpB a reason for delaying them? 

Some other research has been suggested including a more extensive report 
of XSCS development and a comparison of it to other national programs* 
In addition, the information in fhis general summary report could be com- 
pared to staff memoranda, reports to funding agencies, coiranunlcations with 
test center personnel, and reports made at professional Me tings. Such 
comparisons would provide additional information both regarding the design 
and Instruction materials, and In^rmatton as to impact* ^ 
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Infermation from authors, teachers,* and other, st#ff memberB wduld also'be 

^ of interest* In fact, differences In perceptions could provide .gome of 

the most meaningful Informtion for discussion and analysis,. Although 

* ^ -■ * ' . ^ • 

information ■ from^ the ihltial project director is of great value and 

Interests it is easy to see how such perceptions could be slanted, Thil 

makes the report no less validy appropriate, nor significant* 

Certainly -a complete, accoimtlng ©f the development of the ISCS daslpi 
would be a lengthy report— and probably Inappropriate for most journals 
such as the Journal of Research, in Science Teaching, 'jhe article, as it 
appearg, is probably the most that could be expected. However, this 
^ reviewer would favor a morel extensive manuscript— perhaps a book like 
those cited earlier. 



/ 



If Specific data exist that would provide the^: background for some of the 
assertions, this information should be published with the usual fdcts, 
table? J and analyses. If such information has been publislied, it should 
have been noted in this paper. Such reports would add innneasurably to 
the field and provide needed information for decisions concerning instruc- 
tion in science. 

f 
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' . i . ■ 

The study was designed to assess thm relationship between students V self 

ratings of ten self-direction skills and four "independent" variables* 
success in the Intermediate Science Curriculum Study (ISCS) program, level 
on the coursfe (Levels II, or III), school attended, and sex. The ten 
salf-dlrection skills were? (1) operating independently^ (2) seeking 
mswers without assistances (3) using class time effectively, (4) planning 
work, (5) using tasic study skills, (6) doing the activities Independently, 
(7) adapting activities and assignments to needs, (8) working at a pace 
commensurate with perceived ability, (9) using excursions, and (10) collect- 
ing their own laboratory materials. 



Rationale 

Many recently developed science programs, including ISCS, require stu^nts 
to assume an active role In directing their own learning, r is study examlneg 
two basic questions relating to the use of these programs* First, whether 
Students- self'-direction skills improve' as they work through the program., 
tod, second, whether achievement in the program is related to the students - 
self-direction skills. 

The study was not designed to test any theoretical model and^he author does 
not cite any of the related studies in the field of learner-^controlled in- 
struction* 
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Research Desl^ and Procedure / 

■ ■ ■ ■ I I - - — I III — ■ I ^ i 

Thfe study uses a one--shot casfe study design. A total of 1108 junior htgh 
school^ students^ from six schools In the O^^a, Nebraskaj area served as the 
subjects. They were not randomly silected but were the^ students of a group 
of teachers enrolled In an ISCS Insei^i^e course,. Th%535 S^^e seven 
students had about seven months experienq^ with Level I* The 410 grade 
sight students' had all completed Level 1 and had about ^even months exper- 
ience with Level 11*. A highj but unspecif led", proportiort of the 164 grade 
alne students had not completed either Level I Ot II an,d so had only about 
seven fflonths experlefice with ISCS, while the remainder had taken ISCS 
throughout junlbr high school. The author did not separate out these two 
groups of grade nine students In the data analyses* 



The students' salf-'dlrectlon skills were assessed using an author devised 
Self-Directed Rating Scale (SDRS) . The instrument consisted of ten items, 
one for each of the skills listed above, ^e students self rated their own 
skills by indicating on a five^-polnt scale (supported by three behavior 
descriptions) the degree to which they perceived they had attained each 
skMl (1 indicated low ability; 5 indicated high ability), A total self- 
direction score was obtained by adding the ratings assigned to each of the 
ten Items, No Indication of the validity or reliability of the instriment 
was provided. 

Student achievement was pt^ylded by teacher ratings. Teachers identified 
students in the top 15 pejcent^bnd the bottom 15 percent of their classes 



in terms of grades, Thisf provided three achievement grouper the top 15 
percent, the middle 70 percent , and the bottom 15 percent. 

The data were analyzed using ANOVA for the '"k" group comparisons (achieve- 
ment, level, and school groups) and the t-test for the two group comparison 
(sex). For the "k" group comparisons ^ only the ANOVA F scores are provided, 

fit 

Fin d in gj^ t 

Statistically signtf leant differences were found among the high, middle, and 
low achievers for each of the ten skill items and the total self-directednes 
score • Except for one itom (adapting curriculimi) , the high achievers 
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perceived themselves as mote self-directed thm the middle or low achievers, 
^The author hypothesiged that the means ior adapting currlqala are in the 
reverse order because high achievers tend to be mot& confotmlng and are 
r^Juatant to skip activities and/*or asslgnmants, 

T - 

The analysis by levels (time ppant using ISCS) indicated non-'Slgnif leant 
differences^except for the Item relating to the use ''of excursions andNthe 
- total score. In theee cases. the ratios were highest for Level II students* 
followed by those for Levels III and I, author hypotheslEed that the 

jvsfylng experience of the Level III subjects is responsible for the lower 
scores in Level III* " 



The 'analysis by school attended Indicated^ significant differences on the 
total score and all Items except for the one relating to seeking answers 
Independently. The reasons for the differences are not revealed this 
study* 



The analysis by sex indica^d several significant differences* Girls scored 
higher on the total score and Items relating to* using class time , iplannlng 
a work schedule ^ using study skills, pacingi using excursions , and cdllecting 
lab mterlals. Boys scored higher on seeking answers independently and 
adapting curriculum. The authbr hypothaslzed that the boys scored higher 
on these items because society tends to value aggressiveness and independ- 
ence in males . 



Interpr etat Ions 

The author concluded that ^ (1) Buccess in programs like ISCS requires ade- . 
quate self-direction skills, (2) these skills Improve with iq^eased exper- 
ience with the program, (3) school "climate" or ways schools or teachers use 
the program may affect students' perceived self-directedness , and (4) girls 
saw themselves as more sel f^directed than boys. 

The implications di|awn were' (1) that self-^dlrectedness should be assessed 
early in the school ypar to provide dia,gnostic data as a basls\for adapting 
instruction fostering skill development, (2) that school "cliniate" can 



affect the succeds of a program like ISCS, and (3.Vchat >oys are more likely 

* . ■ ■* , " 

to have difficulty with these programs than girls, , « 



ABSTRACTOR'S DIALYSIS 

This Study deals with an important area of educatigfnal research — ^the relation 
ship between measurable student characteristics and the ef f ectivaneas of v 
various instructional methods. This area of research has generatbd* a great 
^ da^ of interest and rea#arch over the years as It Is the key to effect Jj^^e, ^ 
individualization of Instruction* , »Unfortmately, the author does not cite 
any of the previous work done nor appear to build on the previous research 
In independent study, learner-controlled instruction, adapting instruction 
student needs, or self evaluation, , - 

The validity of the study Is undermined by the choice of research design. 
The one^shot case study design is one of the weakest possible designs and 
fails to control for many factors that can Jeopardize the .Internal and 
external validity of an eKperiment. The use of the design is particularly 
Inappropriate for a study that attempts to determine whether students* self- 
direction skills Improve as a fimctlon of time, A time-series design with 
a control group would have been much more appropriate, 

«^ 

Using teachers' reports of grades In 5§CS as the basis for rating student 
Achievement Is somewhat questionable. The Individualized Teacher Prepara- 
tion module Evaluating and Reporting Progress (ISCSj i972)/^Trctm3:ages \ 
teachers to consider such subjective factors asi self-^pa^ing efflelency^^ 
self-reliance, and social responsibility in determining a student's grade. 
The reported relationship between achievement and self-direction skills may 
simply be evidence that teachers are considering these skills in determin- 
ing the grades. 



One of the most serious weaknesses of the s is the lack of any validity 

and reliability data oit the SDRS instrmnent, in the absence of such data 
all of the findings are suspect. The relationship between self-direct iom 
and achievement may be just a function of grading practices or just a "halo" 
effect- The variation among the schools may be more a function of SES than 
"climate*" The higher salf-directedness among girls m^y just be an 
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eacpresalon of the general tendency of junior high age boys to rate themselves 
lower In achieveMnt than girls do (Russell, 1953), ^ The validity of the 
study would have bee^ ImpMved If the Independence scale of one of the stan- 
dard personality tests had been used* If none of the published tests were 
suitable and a new scale had to be created^ th^author should have provided 
SOM lndlca,tlon of the instrument's^ vajldlty and reliability as Para and 
Butiow (1973) and Wang and Stiles (is76) did whan they devised similar Instru- 



Bients, 



The data idobtained from the use of the SDKS representtj at best^ ordmal data. 
Although the \ise pf parametric tests , such as ANOVA/ and the t=test used by ,^ 
the author, to analyze ordinal data is coninon i^^ducatiohal reaearch and 
can, to some extent, be Justified by refaranca to the rigor of these tests ^ 
the use of non-parametric methods would have been more appropriate, 

" / • 

The analysis of the relationship between self-dlrectedness and time in the 
program would have been improved if the Level III students had bean separated 
Into two groups f those in their first year in ISCS and those in their third 
year* This would have allowed comparisons among students with one, twoj and 
three years of axperienca as well as a comparison between Level I students 
and Level II students with one year of experience, ^ 

The manner of presentation of the results of the statistical tests could also 
have beeji improved. Tables I, II, and III list the results of the ANOVA 
tests for differences among groups of students differing with respect to 
achievement (three groups) , level (three groups) , and school (six groups) 
respectively. The F scores listed in these tables indicate only whether or 
not any of the differences between pairs of groups are statistically signi- 
ficant but not whether a^l differences are significant or where the 
differences lie. The failure to Indicate the result of subsequent pair- 
wise comparisons (there is no indication that these were even computed) means 
that it is impossible for a reader to determine which differences are statis^ 
tically significant and which are not. 

The finding of significant variation in self -direct ion skills among students 
in different schools is an interesting one. Although this may be the result 
of SES or school "climate,*^ it may well be due to the fact that the ISCS 
teachers vary considerably In their use of the program, Subsequent 
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Invasclgators would be well advised tpXmftke iSe of the Level of Use model 
(Loucks, Newlove'i and Hall, 1975) or the LeveV of Implementation model 
(Neufeld^ 1978) to attempt to measure and account for this variation* 

Many of the new science programs place a great deal of responsibility on the 
students for managing their oto learning, \Thf& study represents one attempt 
Aq relate student characteristics and program requirements and ^outcomes for 
^eMm Inst rue ttonaL approach* However, much research is still needed to 
determine which students can best benefit from this approach and how to 
optimize student outcomes, , , 
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